Compare commits

..
Author SHA1 Message Date
sergeych 9e11519608 revert to wirking ugly fix for decodeClassObj 2025-12-13 23:11:28 +01:00
sergeych fba44622e5 Refactor toString implementations to support Scope context, add inspect, and improve assertions readability. 2025-12-13 13:48:57 +01:00
sergeych 2737aaa14e add mapNotNull to ObjIterable with documentation 2025-12-12 13:48:02 +01:00
sergeych bce88ced43 Merge pull request 'fix/scope-parent-cycle' (#92) from fix/scope-parent-cycle into master
Reviewed-on: #92
2025-12-11 03:09:45 +03:00
sergeych fd473a32d8 refine some tests 2025-12-11 00:55:05 +01:00
sergeych d15dfb6087 core: prevent scope parent-chain cycles when reusing pooled frames
- Scope.resetForReuse: fully detach before re-parenting (clear state, parent=null, new frameId), then validate with ensureNoCycle and assign parent/args/pos/thisObj
- ScopePool.borrow (JVM/Android/JS/Native/Wasm): defensive fallback to fresh Scope allocation if resetForReuse detects a cycle
- docs: add docs/fix-scope-parent-cycle.md describing the change and expectations
- test: add ScopeCycleRegressionTest to ensure instance method call pattern does not crash and returns "ok"
2025-12-11 00:50:46 +01:00
sergeych b953282251 docs on updated scopes 2025-12-10 00:04:07 +01:00
sergeych bcabfc8962 fix endless recursion in scope resolution in some specific cases 2025-12-09 23:55:50 +01:00
sergeych c0fab3d60e bump version to 1.0.7-SNAPSHOT; fix potential infinite loops in Scope traversal 2025-12-09 07:33:05 +01:00
sergeych 55caa65f97 fixed dependencies for plugin building 2025-12-08 18:59:16 +01:00
sergeych b73891d19b ref $91 trace removed 2025-12-08 18:52:38 +01:00
sergeych 8750040926 fix #91 syntax highlighting of kotlin code blocks on the site 2025-12-08 18:51:55 +01:00
sergeych 708b908415 fix #88 fix #90 plugin with autocompletion 2025-12-08 03:28:27 +01:00
sergeych c35d684df1 started autocompletion in the plugin 2025-12-07 23:06:05 +01:00
sergeych 678cfbf45e fix #85 lynon empty list encoding 2025-12-07 21:50:47 +01:00
sergeych bfffea7e69 fix #83 import-aware quickdocs 2025-12-06 22:25:21 +01:00
sergeych 40f11b6f29 fix #82 refactored and added builtin docs to all symbols 2025-12-06 21:10:40 +01:00
sergeych e25fc95cbf fix #81 site search improved 2025-12-06 18:03:15 +01:00
sergeych a6085b11a1 ignore distributables 2025-12-06 17:37:31 +01:00
sergeych 819fdd82b3 removed distributables from git (derivatives, now there is a copy on the site) 2025-12-06 17:29:11 +01:00
sergeych 2e96d75b9f fix #80 edge case !isSomething() bug fixed
+String.last()
2025-12-06 15:07:38 +01:00
sergeych f616326383 docs on cli tool, restored cli building tools 2025-12-05 21:46:19 +01:00
sergeych 1e2bbe1fc5 fix #79 enum toJson serialization 2025-12-05 21:39:43 +01:00
sergeych b630d69186 fix #78 add fmt CLI subcommand and improve legacy script execution paths 2025-12-05 21:02:18 +01:00
sergeych 20f4e54a02 red #77 tests and docs for jsom map serialization 2025-12-05 15:42:26 +01:00
sergeych e58896f087 red #77 more json docs 2025-12-05 11:57:21 +01:00
sergeych 080eac2e1a fix #77 Instant.toJson 2025-12-05 11:47:42 +01:00
sergeych a31befef0b plugin update 2025-12-05 00:33:20 +01:00
sergeych 65a7555e93 fix $76 add support for enum constants highlighting and initial enum documentation 2025-12-05 00:30:32 +01:00
sergeych 84f2f8fac4 fix #75 simple classes JSON serialization with custom formats 2025-12-04 23:58:31 +01:00
sergeych 0c31ec63ee fix #74 duplicate constructor amd state vars with Lynon serialization 2025-12-04 23:23:31 +01:00
sergeych 603023962e fix $73 reg #74 val assignment bug fix. Also, cosmetics on syntax highlighting 2025-12-04 22:11:49 +01:00
sergeych e765784170 plugin updated to v1.0.5 2025-12-04 17:06:41 +01:00
sergeych 171e413c5f v1.0.5-SNAPSHOT started json and kotlinx serialization support 2025-12-04 17:05:07 +01:00
sergeych 5cfc15cf17 fix #72 comments are allowed in class constructor; some configuration bigs fixed 2025-12-04 16:09:27 +01:00
sergeych b8f27c7a18 migrated stdlib to separate .lyng files and added build-time generation of Kotlin constants. reduced noise in plugin 2025-12-04 12:26:38 +01:00
sergeych 6a6de83972 updated plugin binary 2025-12-03 17:28:44 +01:00
sergeych a3b8dbd9d8 improved annotation caching and error range handling for highlighting 2025-12-03 16:27:05 +01:00
sergeych c63d643469 fixed formatter bugs and some tests; upgraded formatting in plugin 2025-12-03 15:37:35 +01:00
sergeych f592689631 annotation highlighting (missing) 2025-12-03 13:49:27 +01:00
sergeych 834f3118c8 annotation highlighting 2025-12-03 13:49:09 +01:00
sergeych d285335e1c plugin: spell checker f0r 2025 2025-12-03 12:48:06 +01:00
sergeych 2e17297355 Add Grazie-backed grammar checker and dictionary support to Lyng IDEA plugin and variants for replacements with or without Grazie interfaces 2025-12-03 01:29:34 +01:00
sergeych fbea13570e idea plugin now shows errors that prevent syntax highlighting and show cached syntax coloring remembered from the last successive pass 2025-12-02 03:29:14 +01:00
sergeych 067970b80c Add MiniAst documentation system and Markdown rendering for built-in and IDE documentation. 2025-12-02 03:04:14 +01:00
sergeych c52e132dcc more readme 2025-12-01 19:14:43 +01:00
sergeych 53f00e6c6c readme fixes 2025-12-01 18:11:03 +01:00
sergeych ec49bbbf52 idea plugin 0.0.2-SNAPSHOT, improced, added reformat code. Formatting tools improved in lynglib. Site information added 2025-12-01 17:50:27 +01:00
sergeych 06e8e1579d idea plugin 0.0.1-SNAPSHOT: basic coloring and editing aids 2025-11-30 23:57:04 +01:00
sergeych 9c342c5c72 Make lyng-mark white and regenerate 256x256 PNG 2025-11-29 10:54:27 +01:00
sergeych 59055ace8c Add 256x256 PNG export of lyng-mark.svg at site/src/jsMain/resources/icons/lyng-mark-256.png 2025-11-29 10:51:01 +01:00
sergeych 2005f405e4 site icon 2025-11-29 09:58:53 +01:00
sergeych 062f344676 published lyngio to maven, added to docs 2025-11-29 09:02:28 +01:00
sergeych 438e48959e - fixed bug in compiler (rare)
- added lyng.io.fs (multiplatform)
- CLI tools now have access to the filesystem
2025-11-29 00:51:01 +01:00
sergeych 41746f22e5 some more samples 2025-11-28 12:38:43 +01:00
sergeych e584c7aa63 site/docs improvements 2025-11-28 11:25:47 +01:00
sergeych 26ddb94f5d fixed version number 2025-11-27 21:42:42 +01:00
sergeych cbca8cacb5 fix #59 implement iterator cancellation on premature termination (break, exception) and ensure no cancellation on natural completion; add tests 2025-11-27 21:40:02 +01:00
sergeych 8fae4709ed fix #61 set +/- collection gives set with/without items in collection 2025-11-27 20:54:33 +01:00
sergeych d118d29429 map literals 2025-11-27 20:02:19 +01:00
sergeych cb9df79ce3 version bump 2025-11-27 15:24:18 +01:00
sergeych d9a26dd467 better lang page (ribbon & tg channel) 2025-11-27 12:51:32 +01:00
sergeych 813ebebddd small optimizations 2025-11-27 11:55:17 +01:00
sergeych 2d721101dd v1.0.1-SNAPSHOT: named args in calls 2025-11-27 11:24:50 +01:00
sergeych d6e6d68b18 benchmaring tests are now optional 2025-11-27 08:34:49 +01:00
sergeych 83825a9272 fixed call arg precedence bug in last arg callable scenario 2025-11-27 08:15:06 +01:00
sergeych f0fc7ddd84 lyng textmate bundle idea-compatible 2025-11-24 18:33:41 +01:00
sergeych ea0ecb1db3 lynglib: added MiniAST support
lyngweb: better syntax highlighting and code editor
2025-11-24 00:38:17 +01:00
sergeych 28b961d339 Add comprehensive documentation for when statement and integrate related tests and references 2025-11-22 20:11:01 +01:00
sergeych 4d1cd491e0 fixed try ling key processing 2025-11-22 20:01:16 +01:00
sergeych 5fbb1d5393 cosmetics 2025-11-22 15:17:12 +01:00
sergeych 9a4131ee3d lyngweb: refactor SiteHighlight to align package structure and add adjustable textarea height 2025-11-22 14:21:45 +01:00
sergeych 391e200f19 readme updates 2025-11-22 01:41:13 +01:00
sergeych 32e739ab8f Add comprehensive embedding guide for Lyng in Kotlin projects 2025-11-22 01:39:51 +01:00
sergeych f4375ad627 lyngweb: add Maven publishing configuration, range iterability tests, and minor editor refinements 2025-11-22 01:22:39 +01:00
sergeych faead76688 lyngweb: introduce reusable editor and highlighting utilities 2025-11-22 00:34:45 +01:00
sergeych 72c6dc2bde fixed wrong line report on throw statement 2025-11-21 17:18:59 +01:00
sergeych a229f227e1 tryling: better error reporting 2025-11-21 09:44:48 +01:00
sergeych 01632dc6d7 tryling: snart mini-editor 2025-11-21 02:00:01 +01:00
sergeych 4e37d0be26 tryling cosmetics 2025-11-21 00:45:18 +01:00
sergeych fa3fda144b added tryling, v0 2025-11-21 00:38:43 +01:00
sergeych 2b320ab52a restructured site: split to separate sources. Search improved 2025-11-20 19:06:15 +01:00
sergeych f1e978599c deployment refined 2025-11-20 01:17:56 +01:00
sergeych 215c7245a0 Enhance site navigation with improved search highlighting, refined navbar offset handling, query-based search term extraction, and additional docs layout adjustments. Also, add new resources and simplify deployment script paths. 2025-11-20 01:00:32 +01:00
sergeych d307ed2a04 inject back button inline for docs' H1 headers, fetch markdown titles progressively, and add deployment script 2025-11-19 23:56:31 +01:00
sergeych b82af3dceb site search 2025-11-19 23:22:12 +01:00
sergeych 1fadc42414 added DocsPage, improved navbar with dynamic height handling, MathJax integration, new TOC features, and extensive markdown processing 2025-11-19 21:52:58 +01:00
sergeych 918534afb5 fixed highlighting and styles 2025-11-19 17:52:48 +01:00
sergeych 646a676b3e added scrollspy for TOC highlighting, theme toggle support, and dark/light mode handling 2025-11-19 14:23:07 +01:00
sergeych f4d1a77496 added extensive markdown rendering tests and Kotlin/JS externals for marked library 2025-11-18 23:10:22 +01:00
sergeych 67e4d76f59 kotlin upgraded, site started 2025-11-18 00:24:19 +01:00
sergeych beb462fd62 readme fix 2025-11-17 15:45:37 +01:00
sergeych 88f035beb6 version bump 2025-11-17 15:25:26 +01:00
sergeych 41657b3558 another MI bug fixed, added tests for MI serialization 2025-11-17 15:09:02 +01:00
sergeych 1e6dd89778 added test for scope locality 2025-11-17 14:15:38 +01:00
sergeych 6eabcc315f fixed errors 2025-11-17 14:08:32 +01:00
sergeych aeeec2d417 added Multiple Inheritance 2025-11-17 00:28:07 +01:00
sergeych 882df67909 added new performance flags, extended PIC handling to size 4, introduced fast paths for ObjFastIntRangeIterator, small-arity arguments, index access, and unary operations 2025-11-16 00:17:15 +01:00
sergeych 0798bbee9b version bump, added parallel execution tests 2025-11-15 10:45:33 +01:00
sergeych 5df923920c more test for apply 2025-11-12 13:01:35 +01:00
sergeych 76d89b43db added bitwise operators and shift support, updated parser, compiler, and docs. fixed copyright. 2025-11-12 12:57:35 +01:00
sergeych 852383e3b1 fixed execution on JS and native platforms 2025-11-12 10:32:11 +01:00
sergeych 28e8648794 fix to make it build on all platforms 2025-11-11 23:20:22 +01:00
sergeych d2a930c0e8 further optimizations: improve PIC miss caching and add fast paths for string, char, and numeric operations 2025-11-11 22:51:03 +01:00
sergeych 0eea73c118 further optimizations 2025-11-11 22:04:04 +01:00
sergeych fdb056e78e big optimization 2025-11-11 21:22:10 +01:00
sergeych dc3000e9f7 version bump, better diagnostic on unknown object serialization 2025-11-11 15:40:00 +01:00
sergeych e8f0846840 nicrooptimizations 2025-11-10 23:59:51 +01:00
sergeych 2af5852d44 JVM multithreaded scope pool now turned on by default 2025-11-10 23:08:58 +01:00
sergeych 38c1b3c209 big optimization round 2025-11-10 22:14:21 +01:00
sergeych 029fde2883 more optimizations 2025-11-10 02:14:18 +01:00
sergeych 1498140892 further locals optimization 2025-11-10 01:35:15 +01:00
sergeych 0b9e94c6e9 v0.10.1-SNAPSHOT optimization: Accessor refactored now use more effective ObjRef and slots for locals 2025-11-09 23:19:02 +01:00
sergeych 6ca3e4589e fixed errors with some exception object naming and serialization 2025-11-07 02:52:15 +04:00
sergeych 65963ce537 v0.9.3 published for all platforms; dyanimc improvements & more 2025-11-02 14:20:35 +01:00
sergeych 9d222be924 added random bitArray 2025-11-01 22:19:40 +01:00
sergeych f00631cd88 some more docs 2025-11-01 22:10:26 +01:00
sergeych b8d6ff01a6 fixed invoke overriden field problem in dynamic and the dynamic's Lyng class. 2025-11-01 22:07:22 +01:00
sergeych ba725fc9ed less debug noise
open ObjDynamic
found a pfoblem with dynamic, see todo in OOTest.kt
2025-11-01 18:33:37 +01:00
sergeych 5d4d548275 lzw utils 2025-11-01 10:28:04 +01:00
sergeych 7e289382ed Scope.copy renamed to createChild for clarity; fixed error with Exception self serializing. 2025-10-31 14:58:02 +04:00
sergeych c854b06683 fixed comment before else in if;
dynamic now support indexing access
2025-10-04 11:25:19 +04:00
sergeych e8b630715a optimized BitArray.equals 2025-09-26 20:06:54 +04:00
sergeych 3481a718b1 serializable and hashable BitArray. Comparable BitList. Small improvements 2025-09-26 19:45:28 +04:00
sergeych 2696f1546d upgrading to kotlin 2.2 2025-09-26 11:48:40 +04:00
sergeych f45fa7f7a0 fix #49 regular expressions, operator match and docs 2025-09-05 17:53:19 +04:00
sergeych ead2f7168e ref #49 basic regex support (but not in when yet) 2025-08-25 12:34:04 +03:00
sergeych 2743511b62 better readme 2025-08-24 09:04:40 +03:00
sergeych 8431ab4f96 missing link in readme 2025-08-24 00:15:25 +03:00
sergeych 80735f032d more links in docs 2025-08-24 00:10:17 +03:00
sergeych dc837e2095 +shiffle
+List.binarySearch
2025-08-24 00:09:38 +03:00
sergeych 0ec0ed96ee sorted* moved to Iterable
in-place List sort
v0.8.15-SNAPSHOT
2025-08-23 22:53:10 +03:00
sergeych a5e51a3f90 more tests 2025-08-23 09:41:23 +03:00
sergeych cf79163802 reflected current state in readme 2025-08-22 16:18:04 +03:00
sergeych c8e8bdc466 ref #56 exceptions are serializable
+fixed ambigity betwee lyng/kotln toString, added directional parameter to kotlin
2025-08-22 09:57:36 +03:00
sergeych 4b613fda7c fix #58 improper toString overload processing in Lyng
fix #57 Exception#getStackTrace()
ref #56 StackTraceEntry is serializable now
2025-08-22 00:02:23 +03:00
sergeych 8d1cafae80 fix #55 newlines in function declarations and constructors 2025-08-20 11:07:49 +03:00
sergeych 0526cdfb38 fix #54 on current version of the compiler it is no more an issue 2025-08-20 11:00:47 +03:00
sergeych cad6ba936d fix #53 chained calls and multiline expressions 2025-08-20 10:57:15 +03:00
sergeych cb333ab6bd less debug noise 2025-08-18 11:16:57 +03:00
sergeych 23737f9b5c removed debug stuff 2025-08-18 01:48:23 +03:00
sergeych fb6e2aa49e bugfix: elvis + throw 2025-08-18 01:19:49 +03:00
sergeych 95c1da60ed better elvis + throw
+added run {}
2025-08-18 00:56:52 +03:00
sergeych 835333dfad fixed objClass initialization in ObjInt.Zero, added Buffer.toBitInput, minimal descrription of Lynon 2025-08-17 23:51:50 +03:00
sergeych eefecae7b4 more Buffer encodings 2025-08-17 19:26:15 +03:00
sergeych 2ac92a1d09 more elegant Iterable.all 2025-08-17 18:21:52 +03:00
sergeych 464a6dcb99 fixed many bugs with variable visibility in mixed scopes 2025-08-17 12:24:39 +03:00
sergeych eca746b189 commented out delegate creation 2025-08-14 14:34:44 +03:00
sergeych b07452e66e more stdlib and docs, bugfixes 2025-08-14 14:31:51 +03:00
sergeych 202e70a99a less debug noise 2025-08-14 01:36:37 +03:00
sergeych f45310f7d9 default implementation for Iterator.cancelIteration (does nothing) and not implemented errors for iterator methods (better error messages) 2025-08-13 23:34:54 +03:00
sergeych 48a7f0839c less warnings 2025-08-13 23:24:51 +03:00
sergeych 2adb0ff512 +copyrght, apache 2.0 2025-08-13 23:22:04 +03:00
sergeych 6735499959 more enum tests 2025-08-12 14:57:52 +03:00
sergeych b5e89c7e78 smple enums w/serialization 2025-08-12 14:47:41 +03:00
sergeych 84e345b04e more docs on function annotations 2025-08-12 05:01:02 +03:00
sergeych 9bd7aa368e cleanup 2025-08-12 04:52:40 +03:00
sergeych 9704f18284 function annotation and some docs for it 2025-08-12 04:51:26 +03:00
sergeych 804087f16d version bump 2025-08-12 00:16:09 +03:00
sergeych 6d8eed7b8c added "dynamic" fields (get/set/call fields by name using dynamic standard function 2025-08-12 00:15:45 +03:00
sergeych e916d9805a correct end-block token processing (detect more errors) 2025-08-11 22:48:43 +03:00
sergeych c398496ee0 refining the readme/2 2025-08-11 14:41:35 +03:00
sergeych 299738cffd refining the readme 2025-08-11 10:36:29 +03:00
sergeych 62461c09cc readme fix 2025-08-10 23:41:57 +03:00
sergeych 63de82393a multiline string literals 2025-08-10 23:32:03 +03:00
sergeych ed0a21cb06 added test config 2025-08-10 09:51:04 +03:00
sergeych 1baf69f40f more docs on Iterable 2025-08-09 23:23:05 +03:00
sergeych ba8d543d87 more docs on Iterable 2025-08-09 23:20:25 +03:00
sergeych d3785afa6f adopt stdlib to CLI tool 2025-08-09 22:59:43 +03:00
sergeych 3948283481 0.8.4: stadlib in Lyng! added universal Iterable functions 2025-08-09 22:50:48 +03:00
sergeych f9198fe583 parallelism and flows + bugfixes release 2025-08-09 15:55:30 +03:00
sergeych 4917f99197 docs 2025-08-09 15:54:46 +03:00
sergeych e0ed27a01f fixed many bugs in closures processing also in flows 2025-08-09 15:48:41 +03:00
sergeych 9aae33d564 fix #48 flows 2025-08-08 19:33:17 +03:00
sergeych 1a90b25b1e fix #51 ref #48 flows started. bad closure-based bug fixed 2025-08-08 19:23:43 +03:00
382 changed files with 43457 additions and 1709 deletions

No files matched your search

+1
View File
@@ -0,0 +1 @@
*.zip filter=lfs diff=lfs merge=lfs -text
-54
View File
@@ -1,54 +0,0 @@
# This workflow uses actions that are not certified by GitHub.
# They are provided by a third-party and are governed by
# separate terms of service, privacy policy, and support
# documentation.
# This workflow will build a Java project with Gradle and cache/restore any dependencies to improve the workflow execution time
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-java-with-gradle
name: Java CI with Gradle
on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]
workflow_call:
permissions:
contents: read
jobs:
build:
strategy:
matrix:
include:
- target: iosSimulatorArm64Test
os: macos-latest
- target: jvmTest
os: ubuntu-latest
- target: linuxX64Test
os: ubuntu-latest
- target: testDebugUnitTest
os: ubuntu-latest
- target: testReleaseUnitTest
os: ubuntu-latest
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v3
- name: Validate Gradle Wrapper
uses: gradle/actions/wrapper-validation@v3
- uses: actions/cache@v3
with:
path: |
~/.konan
key: ${{ runner.os }}-${{ hashFiles('**/.lock') }}
- name: Set up JDK 17
uses: actions/setup-java@v3
with:
java-version: '17'
distribution: 'temurin'
- name: Build with Gradle
uses: gradle/gradle-build-action@ce999babab2de1c4b649dc15f0ee67e6246c994f
with:
arguments: ${{ matrix.target }}
-25
View File
@@ -1,25 +0,0 @@
name: Publish
on:
release:
types: [released, prereleased]
jobs:
publish:
name: Release build and publish
runs-on: macOS-latest
steps:
- name: Check out code
uses: actions/checkout@v4
- name: Set up JDK 21
uses: actions/setup-java@v4
with:
distribution: 'zulu'
java-version: 21
- name: Publish to MavenCentral
run: ./gradlew publishToMavenCentral --no-configuration-cache
env:
ORG_GRADLE_PROJECT_mavenCentralUsername: ${{ secrets.MAVEN_CENTRAL_USERNAME }}
ORG_GRADLE_PROJECT_mavenCentralPassword: ${{ secrets.MAVEN_CENTRAL_PASSWORD }}
ORG_GRADLE_PROJECT_signingInMemoryKeyId: ${{ secrets.SIGNING_KEY_ID }}
ORG_GRADLE_PROJECT_signingInMemoryKeyPassword: ${{ secrets.SIGNING_PASSWORD }}
ORG_GRADLE_PROJECT_signingInMemoryKey: ${{ secrets.GPG_KEY_CONTENTS }}
+2
View File
@@ -15,3 +15,5 @@ xcuserdata
/kotlin-js-store/yarn.lock
/test.lyng
/sample_texts/1.txt.gz
/kotlin-js-store/wasm/yarn.lock
/distributables
@@ -0,0 +1,28 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="Tests in 'lyng.lynglib.jvmTest'" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value=":lynglib:cleanJvmTest" />
<option value=":lynglib:jvmTest" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<ExternalSystemDebugServerProcess>false</ExternalSystemDebugServerProcess>
<ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
<ExternalSystemDebugDisabled>false</ExternalSystemDebugDisabled>
<DebugAllEnabled>false</DebugAllEnabled>
<RunAsTest>true</RunAsTest>
<GradleProfilingDisabled>false</GradleProfilingDisabled>
<GradleCoverageDisabled>false</GradleCoverageDisabled>
<method v="2" />
</configuration>
</component>
@@ -0,0 +1,24 @@
<component name="ProjectRunConfigurationManager">
<configuration default="false" name="lyng:site [jsBrowserDevelopmentRun]" type="GradleRunConfiguration" factoryName="Gradle">
<ExternalSystemSettings>
<option name="executionName" />
<option name="externalProjectPath" value="$PROJECT_DIR$/site" />
<option name="externalSystemIdString" value="GRADLE" />
<option name="scriptParameters" value="--continuous" />
<option name="taskDescriptions">
<list />
</option>
<option name="taskNames">
<list>
<option value="jsBrowserDevelopmentRun" />
</list>
</option>
<option name="vmOptions" />
</ExternalSystemSettings>
<ExternalSystemDebugServerProcess>true</ExternalSystemDebugServerProcess>
<ExternalSystemReattachDebugProcess>true</ExternalSystemReattachDebugProcess>
<DebugAllEnabled>false</DebugAllEnabled>
<RunAsTest>false</RunAsTest>
<method v="2" />
</configuration>
</component>
+83
View File
@@ -0,0 +1,83 @@
## Changelog
### Unreleased
- Docs: Scopes and Closures guidance
- New page: `docs/scopes_and_closures.md` detailing `ClosureScope` resolution order, recursion‑safe helpers (`chainLookupIgnoreClosure`, `chainLookupWithMembers`, `baseGetIgnoreClosure`), cycle prevention, and capturing lexical environments for callbacks (`snapshotForClosure`).
- Updated: `docs/advanced_topics.md` (link to the new page), `docs/parallelism.md` (closures in `launch`/`flow`), `docs/OOP.md` (visibility from closures with preserved `currentClassCtx`), `docs/exceptions_handling.md` (compatibility alias `SymbolNotFound`).
- Tutorial: added quick link to Scopes and Closures.
- IDEA plugin: Lightweight autocompletion (experimental)
- Global completion: local declarations, in‑scope parameters, imported modules, and stdlib symbols.
- Member completion: after a dot, suggests only members of the inferred receiver type (incl. chained calls like `Path(".." ).lines().` → `Iterator` methods). No global identifiers appear after a dot.
- Inheritance-aware: direct class members first, then inherited (e.g., `List` includes `Collection`/`Iterable` methods).
- Heuristics: handles literals (`"…"` → `String`, numbers → `Int/Real`, `[...]` → `List`, `{...}` → `Dict`) and static `Namespace.` members.
- Performance: capped results, early prefix filtering, per‑document MiniAst cache, cancellation checks.
- Toggle: Settings | Lyng Formatter → "Enable Lyng autocompletion (experimental)" (default ON).
- Stabilization: DEBUG completion/Quick Doc logs are OFF by default; behavior aligned between IDE and isolated engine tests.
- Language: Named arguments and named splats
- New call-site syntax for named arguments using colon: `name: value`.
- Positional arguments must come before named; positionals after a named argument inside parentheses are rejected.
- Trailing-lambda interaction: if the last parameter is already assigned by name (or via a named splat), a trailing `{ ... }` block is illegal.
- Named splats: `...` can now expand a Map into named arguments.
- Only string keys are allowed; non-string keys raise a clear error.
- Duplicate assignment across named args and named splats is an error.
- Ellipsis (variadic) parameters remain positional-only and cannot be named.
- Rationale: `=` is assignment and an expression in Lyng; `:` at call sites avoids ambiguity. Declarations keep `name: Type`; call-site casts continue to use `as` / `as?`.
- Documentation updated: proposals and declaring-arguments sections now cover named args/splats and error cases.
- Tests added covering success cases and errors for named args/splats and trailing-lambda interactions.
- Tooling: Highlighters and TextMate bundle updated for named args
- Website/editor highlighter (lyngweb + site) works with `name: value` and `...Map("k" => v)`; added JS tests covering punctuation/operator spans for `:` and `...`.
- TextMate grammar updated to recognize named call arguments: `name: value` after `(` or `,` with `name` highlighted as `variable.parameter.named.lyng` and `:` as punctuation; excludes `::`.
- TextMate bundle version bumped to 0.0.3; README updated with details and guidance.
- Multiple Inheritance (MI) completed and enabled by default:
- Active C3 Method Resolution Order (MRO) for deterministic, monotonic lookup across complex hierarchies and diamonds.
- Qualified dispatch:
- `this@Type.member(...)` inside class bodies starts lookup at the specified ancestor.
- Cast-based disambiguation: `(expr as Type).member(...)`, `(expr as? Type)?.member(...)` (works with existing safe-call `?.`).
- Field inheritance (`val`/`var`) under MI:
- Instance storage is disambiguated per declaring class; unqualified read/write resolves to the first match in MRO.
- Qualified read/write targets the chosen ancestor’s storage.
- Constructors and initialization:
- Direct bases are initialized left-to-right; each ancestor is initialized at most once (diamond-safe de-duplication).
- Header-specified constructor arguments are passed to direct bases.
- Visibility enforcement under MI:
- `private` visible only inside the declaring class body.
- `protected` visible inside the declaring class and any of its transitive subclasses; unrelated contexts cannot access it (qualification/casts do not bypass).
- Diagnostics improvements:
- Missing member/field messages include receiver class and linearization order; hints for `this@Type` or casts when helpful.
- Invalid `this@Type` reports that the qualifier is not an ancestor and shows the receiver lineage.
- `as`/`as?` cast errors include actual and target type names.
- Documentation updated (docs/OOP.md and tutorial quick-start) to reflect MI with active C3 MRO.
Notes:
- Existing single-inheritance code continues to work; resolution reduces to the single base.
- If code previously relied on non-deterministic parent set iteration, C3 MRO provides a predictable order; disambiguate explicitly if needed using `this@Type`/casts.
# Changelog
All notable changes to this project will be documented in this file.
## Unreleased
- CLI: Added `fmt` as a first-class Clikt subcommand.
- Default behavior: formats files to stdout (no in-place edits by default).
- Options:
- `--check`: check only; print files that would change; exit with code 2 if any changes are needed.
- `-i, --in-place`: write formatted result back to files.
- `--spacing`: apply spacing normalization.
- `--wrap`, `--wrapping`: enable line wrapping.
- Mutually exclusive: `--check` and `--in-place` together now produce an error and exit with code 1.
- Multi-file stdout prints headers `--- <path> ---` per file.
- `lyng --help` shows `fmt`; `lyng fmt --help` displays dedicated help.
- CLI: Preserved legacy script invocation fast-paths:
- `lyng script.lyng [args...]` executes the script directly.
- `lyng -- -file.lyng [args...]` executes a script whose name begins with `-`.
- CLI: Fixed a regression where the root help banner could print before subcommands.
- Root command no longer prints help when a subcommand (e.g., `fmt`) is invoked.
+3
View File
@@ -0,0 +1,3 @@
The Lyng programming language.
Copyright (c) 2024-2025 Sergey Chernov real.sergeych@gmail.com
+75 -30
View File
@@ -1,8 +1,8 @@
# Lyng: modern scripting for kotlin multiplatform
A KMP library and a standalone interpreter
Please visit the project homepage: [https://lynglang.com](https://lynglang.com) and a [telegram channel](https://t.me/lynglang) for updates.
- simple, compact, intuitive and elegant modern code style:
- simple, compact, intuitive and elegant modern code:
```
class Point(x,y) {
@@ -15,23 +15,33 @@ fun swapEnds(first, args..., last, f) {
}
```
- extremely simple Kotlin integration on any platform
- extremely simple Kotlin integration on any platform (JVM, JS, WasmJS, Lunux, MacOS, iOS, Windows)
- 100% secure: no access to any API you didn't explicitly provide
- 100% coroutines! Every function/script is a coroutine, it does not block the thread, no async/await/suspend keyword garbage:
- 100% coroutines! Every function/script is a coroutine, it does not block the thread, no async/await/suspend keyword garbage, see [parallelism]
```
val deferred = launch {
delay(1.5) // coroutine is delayed for 1.5s, thread is not blocked!
"done"
}
// ...
// suspend current coroutine, no thread is blocked again,
// and wait for deferred to return something:
assertEquals("donw", deferred.await())
```
and it is multithreaded on platforms supporting it (automatically, no code changes required, just
`launch` more coroutines and they will be executed concurrently if possible). See [parallelism]
- functional style and OOP together, multiple inheritance, implementing interfaces for existing classes, writing extensions.
- Any unicode letters can be used as identifiers: `assert( sin(π/2) == 1 )`.
- Any Unicode letters can be used as identifiers: `assert( sin(π/2) == 1 )`.
## Resources:
- [Language home](https://lynglang.com)
- [introduction and tutorial](docs/tutorial.md) - start here please
- [Samples directory](docs/samples)
- [Formatter (core + CLI + IDE)](docs/formatter.md)
- [Books directory](docs)
## Integration in Kotlin multiplatform
@@ -84,8 +94,8 @@ import com.sun.source.tree.Scope
import new.sergeych.lyng.*
// simple function
val scope = Scope().apply {
addFn("addArgs") {
val scope = Script.newScope().apply {
addFn("sumOf") {
var sum = 0.0
for (a in args) sum += a.toDouble()
ObjReal(sum)
@@ -96,15 +106,39 @@ val scope = Scope().apply {
// suspend fun doSomeWork(text: String): Int
addFn("doSomeWork") {
// this _is_ a suspend lambda, we can call suspend function,
// and it won't consume the thread:
doSomeWork(args[0].toString()).toObj()
// and it won't consume the thread.
// note that in kotlin handler, `args` is a list of `Obj` arguments
// and return value from this lambda should be Obj too:
doSomeWork(args[0]).toObj()
}
}
// adding constant:
scope.eval("addArgs(1,2,3)") // <- 6
scope.eval("sumOf(1,2,3)") // <- 6
```
Note that the scope stores all changes in it so you can make calls on a single scope to preserve state between calls.
## IntelliJ IDEA plugin: Lightweight autocompletion (experimental)
The IDEA plugin provides a fast, lightweight BASIC completion for Lyng code (IntelliJ IDEA 2024.3+).
What it does:
- Global suggestions: in-scope parameters, same-file declarations (functions/classes/vals), imported modules, and stdlib symbols.
- Member completion after dot: offers only members of the inferred receiver type. It works for chained calls like `Path(".." ).lines().` (suggests `Iterator` methods), and for literals like `"abc".` (String methods) or `[1,2,3].` (List/Iterable methods).
- Inheritance-aware: shows direct class members first, then inherited. For example, `List` also exposes common `Collection`/`Iterable` methods.
- Static/namespace members: `Name.` lists only static members when `Name` is a known class or container (e.g., `Math`).
- Performance: suggestions are capped; prefix filtering is early; parsing is cached; computation is cancellation-friendly.
What it does NOT do (yet):
- No heavy resolve or project-wide indexing. It’s best-effort, driven by a tiny MiniAst + built-in docs registry.
- No control/data-flow type inference.
Enable/disable:
- Settings | Lyng Formatter → "Enable Lyng autocompletion (experimental)" (default: ON).
Tips:
- After a dot, globals are intentionally suppressed (e.g., `lines().Path` is not valid), only the receiver’s members are suggested.
- If completion seems sparse, make sure related modules are imported (e.g., `import lyng.io.fs` so that `Path` and its methods are known).
## Why?
Designed to add scripting to kotlin multiplatform application in easy and efficient way. This is attempt to achieve what Lua is for C/++.
@@ -119,14 +153,13 @@ Designed to add scripting to kotlin multiplatform application in easy and effici
- Javascript, WasmJS, native, JVM, android - batteries included.
- dynamic types in most elegant and concise way
- async, 100% coroutines, supports multiple cores where platofrm supports thread
- async, 100% coroutines, supports multiple cores where platform supports thread
- good for functional an object-oriented style
# Language Roadmap
## v1.0.0
Planned autumn 2025. Complete dynamic language with sufficient standard library:
We are now at **v1.0**: basic optimization performed, battery included: standard library is 90% here, initial
support in HTML, popular editors, and IDEA; tools to syntax highlight and format code are ready. It was released closed to schedule.
Ready features:
@@ -139,31 +172,43 @@ Ready features:
- [x] exception handling: throw, try-catch-finally, exception classes.
- [x] multiplatform maven publication
- [x] documentation for the current state
Under way:
- [x] maps, sets and sequences (flows?)
- [ ] regular exceptions
- [x] modules
- [x] string formatting and tools
- [ ] multiple inheritance for user classes
- [x] launch, deferred, CompletableDeferred, Mutex, etc.
- [ ] site with integrated interpreter to give a try
- [ ] kotlin part public API good docs, integration focused
- [ ] better stack reporting
- [x] multiline strings
- [x] typesafe bit-effective serialization
- [x] compression/decompression (integrated in serialization)
- [x] dynamic fields
- [x] function annotations
- [x] better stack reporting
- [x] regular exceptions + extended `when`
- [x] multiple inheritance for user classes
## v1.1+
Planned features.
## plan: towards v1.5 Enhancing
- [x] site with integrated interpreter to give a try
- [x] kotlin part public API good docs, integration focused
- [ ] type specifications
- [x] Textmate Bundle
- [x] IDEA plugin
- [ ] source docs and maybe lyng.md to a standard
- [ ] macro-style kotlin integration or something else to simplify it
- [ ] metadata first class access from lyng
- [x] aggressive optimizations
- [ ] compile to JVM bytecode optimization
Further
## After 1.5 "Ideal scripting"
- [ ] client with GUI support based on compose multiplatform somehow
- [ ] notebook - style workbooks with graphs, formulaes, etc.
- [ ] language server or compose-based lyng-aware editor
Estimated summer 2026
- propose your feature!
## Authors
@-links are for contacting authors on [project home](https://gitea.sergeych.net/SergeychWorks/lyng): this simplest s to open issue for the person you need to convey any information about this project.
__Sergey Chernov__ @sergeych: Initial idea and architecture, language concept, design, implementation.
__Yulia Nezhinskaya__ @AlterEgoJuliaN: System analysis, math and features design.
[parallelism]: docs/parallelism.md
+167
View File
@@ -0,0 +1,167 @@
#!/bin/bash
#
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
function checkState() {
if [[ $? != 0 ]]; then
echo
echo -- rsync failed. deploy was not finished. deployed version has not been affected
echo
exit 100
fi
}
# Update docs/idea_plugin.md to point to the latest built IDEA plugin zip
# from ./distributables before building the site. The change is temporary and
# the original file is restored right after the build.
DOC_IDEA_PLUGIN="docs/idea_plugin.md"
DOC_IDEA_PLUGIN_BACKUP="${DOC_IDEA_PLUGIN}.deploy_backup"
function updateIdeaPluginDownloadLink() {
if [[ ! -f "$DOC_IDEA_PLUGIN" ]]; then
echo "WARN: $DOC_IDEA_PLUGIN not found; skipping plugin link update"
return 0
fi
# Find the most recently modified plugin zip
local latest
latest=$(ls -t distributables/lyng-idea-*.zip 2>/dev/null | head -n 1)
if [[ -z "$latest" ]]; then
echo "WARN: no distributables/lyng-idea-*.zip found; leaving $DOC_IDEA_PLUGIN unchanged"
return 0
fi
local base
base=$(basename "$latest")
local version
version="${base#lyng-idea-}"
version="${version%.zip}"
local url
url="https://lynglang.com/distributables/${base}"
local newline
newline="### [Download plugin v${version}](${url})"
# Backup and rewrite the specific markdown line if present
cp "$DOC_IDEA_PLUGIN" "$DOC_IDEA_PLUGIN_BACKUP" || {
echo "ERROR: can't backup $DOC_IDEA_PLUGIN"; return 1; }
# Replace the line that starts with the download header; if not found, append it
awk -v repl="$newline" 'BEGIN{done=0} \
/^### \[Download plugin v/ { print repl; done=1; next } \
{ print } \
END { if (done==0) exit 42 }' "$DOC_IDEA_PLUGIN_BACKUP" > "$DOC_IDEA_PLUGIN"
local rc=$?
if [[ $rc -eq 42 ]]; then
echo "WARN: download link not found in $DOC_IDEA_PLUGIN; appending generated link"
echo >> "$DOC_IDEA_PLUGIN"
echo "$newline" >> "$DOC_IDEA_PLUGIN"
elif [[ $rc -ne 0 ]]; then
echo "ERROR: failed to update $DOC_IDEA_PLUGIN; restoring original"
mv -f "$DOC_IDEA_PLUGIN_BACKUP" "$DOC_IDEA_PLUGIN" 2>/dev/null
return 1
fi
}
# default target settings
case "com" in
com)
SSH_HOST=sergeych@lynglang.com # host to deploy to
SSH_PORT=22 # ssh port on it
ROOT=/bigstore/sergeych_pub/lyng # directory to rsync to
;;
# com)
# SSH_HOST=vvk@front-01.neurodatalab.com
# ROOT=/home/vvk
# ;;
*)
echo "*** ERROR: target not specified (use deploy com | dev)"
echo "*** stop"
exit 101
esac
die() { echo "ERROR: $*" 1>&2 ; exit 1; }
# Update the IDEA plugin download link in docs (temporarily), then build, then restore the doc
updateIdeaPluginDownloadLink || echo "WARN: proceeding without updating IDEA plugin download link"
./gradlew site:clean site:jsBrowserDistribution
BUILD_RC=$?
# Always restore original doc if backup exists
if [[ -f "$DOC_IDEA_PLUGIN_BACKUP" ]]; then
mv -f "$DOC_IDEA_PLUGIN_BACKUP" "$DOC_IDEA_PLUGIN"
fi
if [[ $BUILD_RC -ne 0 ]]; then
echo
echo -- build failed. deploy aborted.
echo
exit 100
fi
#exit 0
# Prepare working dir
ssh -p ${SSH_PORT} ${SSH_HOST} "
cd ${ROOT}
rm -rd build 2>/dev/null
if [ -d release ]; then
echo copying current release
cp -r release build
else
echo creating first release
mkdir release
mkdir build
fi
";
# sync files
SRC=./site/build/dist/js/productionExecutable
rsync -e "ssh -p ${SSH_PORT}" -avz -r -d --delete ${SRC}/* ${SSH_HOST}:${ROOT}/build/dist
checkState
#rsync -e "ssh -p ${SSH_PORT}" -avz ./static/* ${SSH_HOST}:${ROOT}/build/dist
#checkState
rsync -e "ssh -p ${SSH_PORT}" -avz -r -d --delete distributables/* ${SSH_HOST}:${ROOT}/build/dist/distributables
checkState
echo
echo finalizing the deploy...
ssh -p ${SSH_PORT} ${SSH_HOST} "
cd ${ROOT}
rm -rd release~
mv release release~
mv build release
cd release
# in this project we needn't restart back when we deploy the front
# ~/bin/restart_service
";
if [[ $? != 0 ]]; then
echo
echo -- finalization failed. the rease might be corrupt. rolling back is not yet implemented.
echo
exit 100
fi
echo
echo Deploy successful
echo
+17
View File
@@ -1,5 +1,22 @@
#!/bin/bash
#
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
set -e
root=./lyng/build/install/lyng-jvm/
+20
View File
@@ -1,5 +1,22 @@
#!/bin/bash
#
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
set -e
file=./lyng/build/bin/linuxX64/releaseExecutable/lyng.kexe
@@ -8,3 +25,6 @@ file=./lyng/build/bin/linuxX64/releaseExecutable/lyng.kexe
strip $file
upx $file
cp $file ~/bin/lyng
cp $file ./distributables/lyng
zip ./distributables/lyng-linuxX64 ./distributables/lyng
rm ./distributables/lyng
+26
View File
@@ -1,5 +1,31 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
plugins {
alias(libs.plugins.androidLibrary) apply false
alias(libs.plugins.kotlinMultiplatform) apply false
alias(libs.plugins.vanniktech.mavenPublish) apply false
}
// Convenience alias to run the IntelliJ IDE with the Lyng plugin from the project root.
// Usage: ./gradlew runIde
// It simply delegates to :lyng-idea:runIde provided by the Gradle IntelliJ Plugin.
tasks.register<org.gradle.api.DefaultTask>("runIde") {
group = "intellij"
description = "Run IntelliJ IDEA with the Lyng plugin (:lyng-idea)"
dependsOn(":lyng-idea:runIde")
}
+37
View File
@@ -0,0 +1,37 @@
# Array
It's an interface if the [Collection] that provides indexing access, like `array[3] = 0`.
Array therefore implements [Iterable] too. The well known implementatino of the `Array` is
[List].
Array adds the following methods:
## Binary search
When applied to sorted arrays, binary search allow to quicly find an index of the element in the array, or where to insert it to keep order:
val coll = [1,2,3,4,5]
assertEquals( 2, coll.binarySearch(3) )
assertEquals( 0, coll.binarySearch(1) )
assertEquals( 4, coll.binarySearch(5) )
val src = (1..50).toList().shuffled()
val result = []
for( x in src ) {
val i = result.binarySearch(x)
assert( i < 0 )
result.insertAt(-i-1, x)
}
assertEquals( src.sorted(), result )
>>> void
So `binarySearch(x)` returns:
- index of `x`, a non-negative number
- negative: `x` not found, but if inserted at position `-returnedValue-1` will leave array sorted.
To pre-sort and array use `Iterable.sorted*` or in-place `List.sort*` families, see [List] and [Iterable] docs.
[Collection]: Collection.md
[Iterable]: Iterable.md
[List]: List.md
+36 -6
View File
@@ -102,20 +102,50 @@ As with [List], it is possible to use ranges as indexes to slice a Buffer:
>>> void
## Encoding
You can encode `String` to buffer using buffer constructor, as was shown. Also, buffer supports out of the box base64 (
which is used in `toString`) and hex encoding:
import lyng.buffer
// to UTF8 and back:
val b = Buffer("hello")
assertEquals( "hello", b.decodeUtf8() )
// to base64 and back:
assertEquals( b, Buffer.decodeBase64(b.base64) )
assertEquals( b, Buffer.decodeHex(b.hex) )
>>> void
## Members
| name | meaning | type |
|---------------|------------------------------------|---------------|
| `size` | size | Int |
| `decodeUtf8` | decodee to String using UTF8 rules | Any |
| `+` | buffer concatenation | Any |
| `toMutable()` | create a mutable copy | MutableBuffer |
| name | meaning | type |
|----------------------------|-----------------------------------------|---------------|
| `size` | size | Int |
| `decodeUtf8` | decode to String using UTF8 rules | Any |
| `+` | buffer concatenation | Any |
| `toMutable()` | create a mutable copy | MutableBuffer |
| `hex` | encode to hex strign | String |
| `Buffer.decodeHex(hexStr) | decode hex string | Buffer |
| `base64` | encode to base64 (url flavor) (2) | String |
| `Buffer.decodeBase64(str)` | decode base64 to new Buffer (2) | Buffer |
| `toBitInput()` | create bit input from a byte buffer (3) | |
(1)
: optimized implementation that override `Iterable` one
(2)
: base64url alphabet is used without trailing '=', which allows string to be used in URI without escaping. Note that
decoding supports both traditional and URL alphabets automatically, and ignores filling `=` characters. Base64URL is
well known and mentioned in the internet, for example, [here](https://base64.guru/standards/base64url).
(3)
: `BitInput` is a bit buffer that is used, for example, in [Lynon.decode](serialization.md)
Also, it inherits methods from [Iterable] and [Array].
[Range]: Range.md
[Iterable]: Iterable.md
+6
View File
@@ -6,6 +6,12 @@ Is a [Iterable] with known `size`, a finite [Iterable]:
val size
}
| name | description |
|------------------------|------------------------------------------------------|
(1)
: `comparator(a,b)` should return -1 if `a < b`, +1 if `a > b` or zero.
See [List], [Set] and [Iterable]
[Iterable]: Iterable.md
+109 -17
View File
@@ -1,5 +1,15 @@
# Iterable interface
The interface for anything that can be iterated, e.g. finite or infinite ordered set of data that can be accessed
sequentially. Almost any data container in `Lyng` implements it: `List`, `Set`, `Buffer`, `RingBuffer`, `BitBuffer`,
`Range` and many others are `Iterable`, also `Collection` and `Array` interfaces inherit it.
`Map` and `String` have `Iterable` members to access its contents too.
Please see also [Collection] interface: many iterables are also collections, and it adds important features.
## Definition:
Iterable is a class that provides function that creates _the iterator_:
class Iterable {
@@ -17,23 +27,99 @@ Iterator itself is a simple interface that should provide only to method:
Just remember at this stage typed declarations are not yet supported.
Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions available:
Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions
available, for example
## Instance methods
val r = 1..10 // Range is Iterable!
assertEquals( [9,10], r.takeLast(2).toList() )
assertEquals( [1,2,3], r.take(3).toList() )
assertEquals( [9,10], r.drop(8).toList() )
assertEquals( [1,2], r.dropLast(8).toList() )
>>> void
| fun/method | description |
|-----------------|---------------------------------------------------------------------------------|
| toList() | create a list from iterable |
| toSet() | create a set from iterable |
| contains(i) | check that iterable contains `i` |
| `i in iterator` | same as `contains(i)` |
| isEmpty() | check iterable is empty |
| forEach(f) | call f for each element |
| toMap() | create a map from list of key-value pairs (arrays of 2 items or like) |
| map(f) | create a list of values returned by `f` called for each element of the iterable |
| indexOf(i) | return index if the first encounter of i or a negative value if not found |
| associateBy(kf) | create a map where keys are returned by kf that will be called for each element |
## joinToString
This methods convert any iterable to a string joining string representation of each element, optionally transforming it
and joining using specified suffix.
Iterable.joinToString(suffux=' ', transform=null)
- if `Iterable` `isEmpty`, the empty string `""` is returned.
- `suffix` is inserted between items when there are more than one.
- `transform` of specified is applied to each element, otherwise its `toString()` method is used.
Here is the sample:
assertEquals( (1..3).joinToString(), "1 2 3")
assertEquals( (1..3).joinToString(":"), "1:2:3")
assertEquals( (1..3).joinToString { it * 10 }, "10 20 30")
>>> void
## `sum` and `sumBy`
These, again, does the thing:
assertEquals( 6, [1,2,3].sum() )
assertEquals( 12, [1,2,3].sumOf { it*2 } )
// sum of empty collections is null:
assertEquals( null, [].sum() )
assertEquals( null, [].sumOf { 2*it } )
>>> void
## map and mapNotNull
Used to transform either the whole iterable stream or also skipping som elements from it:
val source = [1,2,3,4]
// transform every element to string or null:
assertEquals(["n1", "n2", null, "n4"], source.map { if( it == 3 ) null else "n"+it } )
// transform every element to stirng, skipping 3:
assertEquals(["n1", "n2", "n4"], source.mapNotNull { if( it == 3 ) null else "n"+it } )
>>> void
## Instance methods:
| fun/method | description |
|------------------------|---------------------------------------------------------------------------------|
| toList() | create a list from iterable |
| toSet() | create a set from iterable |
| contains(i) | check that iterable contains `i` |
| `i in iterator` | same as `contains(i)` |
| isEmpty() | check iterable is empty |
| forEach(f) | call f for each element |
| toMap() | create a map from list of key-value pairs (arrays of 2 items or like) |
| map(f) | create a list of values returned by `f` called for each element of the iterable |
| indexOf(i) | return index if the first encounter of i or a negative value if not found |
| associateBy(kf) | create a map where keys are returned by kf that will be called for each element |
| first | first element (1) |
| last | last element (1) |
| take(n) | return [Iterable] of up to n first elements |
| taleLast(n) | return [Iterable] of up to n last elements |
| drop(n) | return new [Iterable] without first n elements |
| dropLast(n) | return new [Iterable] without last n elements |
| sum() | return sum of the collection applying `+` to its elements (3) |
| sumOf(predicate) | sum of the modified collection items (3) |
| sorted() | return [List] with collection items sorted naturally |
| sortedWith(comparator) | sort using a comparator that compares elements (1) |
| sortedBy(predicate) | sort by comparing results of the predicate function |
| joinToString(s,t) | convert iterable to string, see (2) |
| reversed() | create a list containing items from this in reverse order |
| shuffled() | create a listof shiffled elements |
(1)
: throws `NoSuchElementException` if there is no such element
(2)
: `joinToString(suffix=" ",transform=null)`: suffix is inserted between items if there are more than one, trasnfom is
optional function applied to each item that must return result string for an item, otherwise `item.toString()` is used.
(3)
: sum of empty collection is `null`
fun Iterable.toList(): List
fun Iterable.toSet(): Set
@@ -44,8 +130,7 @@ Having `Iterable` in base classes allows to use it in for loop. Also, each `Iter
fun Iterable.map(block: (Any?)->Void ): List
fun Iterable.associateBy( keyMaker: (Any?)->Any): Map
## Abstract methods
## Abstract methods:
fun iterator(): Iterator
@@ -57,7 +142,14 @@ Creates a list by iterating to the end. So, the Iterator should be finite to be
## Implemented in classes:
- [List], [Range]
- [List], [Range], [Buffer](Buffer.md), [BitBuffer], [Buffer], [Set], [RingBuffer]
[Collection]: Collection.md
[List]: List.md
[Range]: Range.md
[Set]: Set.md
[RingBuffer]: RingBuffer.md
+3
View File
@@ -9,6 +9,9 @@ To implement the iterator you need to implement only two abstract methods:
### hasNext(): Bool
// lets test
// offset
Should return `true` if call to `next()` will return valid next element.
### next(): Obj
+49 -16
View File
@@ -24,7 +24,7 @@ There is a shortcut for the last:
val list = [10, 20, 30]
[list.last, list.lastIndex]
>>> [30, 2]
>>> [30,2]
__Important__ negative indexes works wherever indexes are used, e.g. in insertion and removal methods too.
@@ -92,22 +92,45 @@ Open end ranges remove head and tail elements:
assert( [1, 2, 3] !== [1, 2, 3])
>>> void
## In-place sort
List could be sorted in place, just like [Collection] provide sorted copies, in a very like way:
val l1 = [6,3,1,9]
l1.sort()
assertEquals( [1,3,6,9], l1)
l1.sortBy { -it }
assertEquals( [1,3,6,9].reversed(), l1)
l1.sort() // 1 3 6 9
l1.sortBy { it % 4 }
// 1,3,6,9 gives, mod 4:
// 1 3 2 1
// we hope we got it also stable:
assertEquals( [1,9,6,3], l1)
>>> void
## Members
| name | meaning | type |
|-------------------------------|---------------------------------------|-------------|
| `size` | current size | Int |
| `add(elements...)` | add one or more elements to the end | Any |
| `insertAt(index,elements...)` | insert elements at position | Int, Any |
| `removeAt(index)` | remove element at position | Int |
| `remove(from,toNonInclusive)` | remove range from (incl) to (nonincl) | Int, Int |
| `remove(Range)` | remove range | Range |
| `removeLast()` | remove last element | |
| `removeLast(n)` | remove n last elements | Int |
| `contains(element)` | check the element is in the list (1) | |
| `[index]` | get or set element at index | Int |
| `[Range]` | get slice of the array (copy) | Range |
| `+=` | append element(s) | List or Obj |
| name | meaning | type |
|-------------------------------|----------------------------------------------|-------------|
| `size` | current size | Int |
| `add(elements...)` | add one or more elements to the end | Any |
| `insertAt(index,elements...)` | insert elements at position | Int, Any |
| `removeAt(index)` | remove element at position | Int |
| `remove(from,toNonInclusive)` | remove range from (incl) to (nonincl) | Int, Int |
| `remove(Range)` | remove range | Range |
| `removeLast()` | remove last element | |
| `removeLast(n)` | remove n last elements | Int |
| `contains(element)` | check the element is in the list (1) | |
| `[index]` | get or set element at index | Int |
| `[Range]` | get slice of the array (copy) | Range |
| `+=` | append element(s) (2) | List or Obj |
| `sort()` | in-place sort, natural order | void |
| `sortBy(predicate)` | in-place sort bu `predicate` call result (3) | void |
| `sortWith(comparator)` | in-place sort using `comarator` function (4) | void |
| `shiffle()` | in-place shiffle contents | |
(1)
: optimized implementation that override `Array` one
@@ -116,7 +139,16 @@ Open end ranges remove head and tail elements:
: `+=` append either a single element, or all elements if the List or other Iterable
instance is appended. If you want to append an Iterable object itself, use `add` instead.
It inherits from [Iterable] too.
(3)
: predicate is called on each element, and the returned values are used to sort in natural
order, e.g. is same as `list.sortWith { a,b -> predicate(a) <=> predicate(b) }`
(4)
: comparator callable takes tho arguments and must return: negative value when first is less,
positive if first is greater, and zero if they are equal. For example, the equvalent comparator
for `sort()` will be `sort { a, b -> a <=> b }
It inherits from [Iterable] too and thus all iterable methods are applicable to any list.
## Member inherited from Array
@@ -134,4 +166,5 @@ It inherits from [Iterable] too.
[Range]: Range.md
[Iterable]: Iterable.md
+73 -8
View File
@@ -1,19 +1,22 @@
# Map
Map is a mutable collection of key-value pars, where keys are unique. Maps could be created with
constructor or `.toMap` methods. When constructing from a list, each list item must be a [Collection] with exactly 2 elements, for example, a [List].
Map is a mutable collection of key-value pairs, where keys are unique. You can create maps in two ways:
- with the constructor `Map(...)` or `.toMap()` helpers; and
- with map literals using braces: `{ "key": value, id: expr, id: }`.
When constructing from a list, each list item must be a [Collection] with exactly 2 elements, for example, a [List].
Important thing is that maps can't contain `null`: it is used to return from missing elements.
Constructed map instance is of class `Map` and implements `Collection` (and therefore `Iterable`)
Constructed map instance is of class `Map` and implements `Collection` (and therefore `Iterable`).
val map = Map( "foo" => 1, "bar" => "buzz" )
assert(map is Map)
assert(map.size == 2)
assert(map is Iterable)
val oldForm = Map( "foo" => 1, "bar" => "buzz" )
assert(oldForm is Map)
assert(oldForm.size == 2)
assert(oldForm is Iterable)
>>> void
Notice usage of the `=>` operator that creates `MapEntry`, which implements also [Collection] of
Notice usage of the `=>` operator that creates `MapEntry`, which also implements [Collection] of
two items, first, at index zero, is a key, second, at index 1, is the value. You can use lists too.
Map keys could be any objects (hashable, e.g. with reasonable hashCode, most of standard types are). You can access elements with indexing operator:
@@ -29,6 +32,36 @@ Map keys could be any objects (hashable, e.g. with reasonable hashCode, most of
assert( map["foo"] == -1)
>>> void
## Map literals { ... }
Lyng supports JavaScript-like map literals. Keys can be string literals or identifiers, and there is a handy identifier shorthand:
- String key: `{ "a": 1 }`
- Identifier key: `{ foo: 2 }` is the same as `{ "foo": 2 }`
- Identifier shorthand: `{ foo: }` is the same as `{ "foo": foo }`
Access uses brackets: `m["a"]`.
val x = 10
val y = 10
val m = { "a": 1, x: x * 2, y: }
assertEquals(1, m["a"]) // string-literal key
assertEquals(20, m["x"]) // identifier key
assertEquals(10, m["y"]) // identifier shorthand expands to y: y
>>> void
Trailing commas are allowed for nicer diffs and multiline formatting:
val m = {
"a": 1,
b: 2,
}
assertEquals(1, m["a"])
assertEquals(2, m["b"])
>>> void
Empty `{}` is reserved for blocks/lambdas; use `Map()` for an empty map.
To remove item from the collection. use `remove`. It returns last removed item or null. Be careful if you
hold nulls in the map - this is not a recommended practice when using `remove` returned value. `clear()`
removes all.
@@ -110,4 +143,36 @@ is equal.
assert( m1 != m3 )
>>> void
## Spreads and merging
Inside map literals you can spread another map with `...` and items will be merged left-to-right; rightmost wins:
val base = { a: 1, b: 2 }
val m = { a: 0, ...base, b: 3, c: 4 }
assertEquals(1, m["a"]) // base overwrites a:0
assertEquals(3, m["b"]) // literal overwrites spread
assertEquals(4, m["c"]) // new key
>>> void
Maps and entries can also be merged with `+` and `+=`:
val m1 = ("x" => 1) + ("y" => 2)
assertEquals(1, m1["x"])
assertEquals(2, m1["y"])
val m2 = { "a": 10 } + ("b" => 20)
assertEquals(10, m2["a"])
assertEquals(20, m2["b"])
var m3 = { a: 1 }
m3 += ("b" => 2)
assertEquals(1, m3["a"])
assertEquals(2, m3["b"])
>>> void
Notes:
- Map literals always use string keys (identifier keys are converted to strings).
- Spreads inside map literals and `+`/`+=` merges require string keys on the right-hand side; this aligns with named-argument splats.
- When you need computed or non-string keys, use the constructor form `Map(...)` or build entries with `=>` and then merge.
[Collection](Collection.md)
+301 -2
View File
@@ -1,6 +1,10 @@
# OO implementation in Lyng
# Object-oriented programming
## Declaration
[//]: # (topMenu)
Lyng supports first class OOP constructs, based on classes with multiple inheritance.
## Class Declaration
The class clause looks like
@@ -71,6 +75,173 @@ Functions defined inside a class body are methods, and unless declared
void
>>> void
## Multiple Inheritance (MI)
Lyng supports declaring a class with multiple direct base classes. The syntax is:
```
class Foo(val a) {
var tag = "F"
fun runA() { "ResultA:" + a }
fun common() { "CommonA" }
private fun privateInFoo() {}
protected fun protectedInFoo() {}
}
class Bar(val b) {
var tag = "B"
fun runB() { "ResultB:" + b }
fun common() { "CommonB" }
}
// Multiple inheritance with per‑base constructor arguments
class FooBar(a, b) : Foo(a), Bar(b) {
// You can disambiguate via qualified this or casts
fun fromFoo() { this@Foo.common() }
fun fromBar() { this@Bar.common() }
}
val fb = FooBar(1, 2)
assertEquals("ResultA:1", fb.runA())
assertEquals("ResultB:2", fb.runB())
// Unqualified ambiguous member resolves to the first base (leftmost)
assertEquals("CommonA", fb.common())
// Disambiguation via casts
assertEquals("CommonB", (fb as Bar).common())
assertEquals("CommonA", (fb as Foo).common())
// Field inheritance with name collisions
assertEquals("F", fb.tag) // unqualified: leftmost base
assertEquals("F", (fb as Foo).tag) // qualified read: Foo.tag
assertEquals("B", (fb as Bar).tag) // qualified read: Bar.tag
fb.tag = "X" // unqualified write updates leftmost base
assertEquals("X", (fb as Foo).tag)
assertEquals("B", (fb as Bar).tag)
(fb as Bar).tag = "Y" // qualified write updates Bar.tag
assertEquals("X", (fb as Foo).tag)
assertEquals("Y", (fb as Bar).tag)
```
Key rules and features:
- Syntax
- `class Derived(args) : Base1(b1Args), Base2(b2Args)`
- Each direct base may receive constructor arguments specified in the header. Only direct bases receive header args; indirect bases must either be default‑constructible or receive their args through their direct child (future extensions may add more control).
- Resolution order (C3 MRO — active)
- Member lookup is deterministic and follows C3 linearization (Python‑like), which provides a monotonic, predictable order for complex hierarchies and diamonds.
- Intuition: for `class D() : B(), C()` where `B()` and `C()` both derive from `A()`, the C3 order is `D → B → C → A`.
- The first visible match along this order wins.
- Qualified dispatch
- Inside a class body, use `this@Type.member(...)` to start lookup at the specified ancestor.
- For arbitrary receivers, use casts: `(expr as Type).member(...)` or `(expr as? Type)?.member(...)` (safe‑call `?.` is already available in Lyng).
- Qualified access does not relax visibility.
- Field inheritance (`val`/`var`) and collisions
- Instance storage is kept per declaring class, internally disambiguated; unqualified read/write resolves to the first match in the resolution order (leftmost base).
- Qualified read/write (via `this@Type` or casts) targets the chosen ancestor’s storage.
- `val` remains read‑only; attempting to write raises an error as usual.
- Constructors and initialization
- During construction, direct bases are initialized left‑to‑right in the declaration order. Each ancestor is initialized at most once (diamond‑safe de‑duplication).
- Arguments in the header are evaluated in the instance scope and passed to the corresponding direct base constructor.
- The most‑derived class’s constructor runs after the bases.
- Visibility
- `private`: accessible only inside the declaring class body; not visible in subclasses and cannot be accessed via `this@Type` or casts.
- `protected`: accessible in the declaring class and in any of its transitive subclasses (including MI), but not from unrelated contexts; qualification/casts do not bypass it.
- Diagnostics
- When a member/field is not found, error messages include the receiver class name and the considered linearization order, with suggestions to disambiguate using `this@Type` or casts if appropriate.
- Qualifying with a non‑ancestor in `this@Type` reports a clear error mentioning the receiver lineage.
- `as`/`as?` cast errors mention the actual and target types.
Compatibility notes:
- Existing single‑inheritance code continues to work unchanged; its resolution order reduces to the single base.
- If your previous code accidentally relied on non‑deterministic parent set iteration, it may change behavior — the new deterministic order is a correctness fix.
### Migration note (declaration‑order → C3)
Earlier drafts and docs described a declaration‑order depth‑first linearization. Lyng now uses C3 MRO for member lookup and disambiguation. Most code should continue to work unchanged, but in rare edge cases involving diamonds or complex multiple inheritance, the chosen base for an ambiguous member may change to reflect C3. If needed, disambiguate explicitly using `this@Type.member(...)` inside class bodies or casts `(expr as Type).member(...)` from outside.
## Enums
Lyng provides lightweight enums for representing a fixed set of named constants. Enums are classes whose instances are predefined and singletons.
Current syntax supports simple enum declarations with just entry names:
enum Color {
RED, GREEN, BLUE
}
Usage:
- Type of entries: every entry is an instance of its enum type.
assert( Color.RED is Color )
- Order and names: each entry has zero‑based `ordinal` and string `name`.
assertEquals(0, Color.RED.ordinal)
assertEquals("BLUE", Color.BLUE.name)
- All entries as a list in declaration order: `EnumType.entries`.
assertEquals([Color.RED, Color.GREEN, Color.BLUE], Color.entries)
- Lookup by name: `EnumType.valueOf("NAME")` → entry.
assertEquals(Color.GREEN, Color.valueOf("GREEN"))
- Equality and comparison:
- Equality uses identity of entries, e.g., `Color.RED == Color.valueOf("RED")`.
- Cross‑enum comparisons are not allowed.
- Ordering comparisons use `ordinal`.
assert( Color.RED == Color.valueOf("RED") )
assert( Color.RED.ordinal < Color.BLUE.ordinal )
>>> void
### Enums with `when`
Use `when(subject)` with equality branches for enums. See full `when` guide: [The `when` statement](when.md).
enum Color { RED, GREEN, BLUE }
fun describe(c) {
when(c) {
Color.RED, Color.GREEN -> "primary-like"
Color.BLUE -> "blue"
else -> "unknown" // if you pass something that is not a Color
}
}
assertEquals("primary-like", describe(Color.RED))
assertEquals("blue", describe(Color.BLUE))
>>> void
### Serialization
Enums are serialized compactly with Lynon: the encoded value stores just the entry ordinal within the enum type, which is both space‑efficient and fast.
import lyng.serialization
enum Color { RED, GREEN, BLUE }
val e = Lynon.encode(Color.BLUE)
val decoded = Lynon.decode(e)
assertEquals(Color.BLUE, decoded)
>>> void
Notes and limitations (current version):
- Enum declarations support only simple entry lists: no per‑entry bodies, no custom constructors, and no user‑defined methods/fields on the enum itself yet.
- `name` and `ordinal` are read‑only properties of an entry.
- `entries` is a read‑only list owned by the enum type.
## fields and visibility
It is possible to add non-constructor fields:
@@ -130,6 +301,25 @@ Private fields are visible only _inside the class instance_:
void
>>> void
### Protected members
Protected members are available to the declaring class and all of its transitive subclasses (including via MI), but not from unrelated contexts:
```
class A() {
protected fun ping() { "pong" }
}
class B() : A() {
fun call() { this@A.ping() }
}
val b = B()
assertEquals("pong", b.call())
// Unrelated access is forbidden, even via cast
assertThrows { (b as A).ping() }
```
It is possible to provide private constructor parameters so they can be
set at construction but not available outside the class:
@@ -235,6 +425,110 @@ as they are modifying the type, not the context.
Beware of it. We might need to reconsider it later.
## dynamic symbols
Sometimes it is convenient to provide methods and variables whose names are not known at compile time. For example, it could be external interfaces not known to library code, user-defined data fields, etc. You can use `dynamic` function to create such:
// val only dynamic object
val accessor = dynamic {
// all symbol reads are redirected here:
get { name ->
// lets provide one dynamic symbol:
if( name == "foo" ) "bar" else null
// consider also throw SymbolNotDefinedException
}
}
// now we can access dynamic "fields" of accessor:
assertEquals("bar", accessor.foo)
assertEquals(null, accessor.bar)
>>> void
The same we can provide writable dynamic fields (var-type), adding set method:
// store one dynamic field here
var storedValueForBar = null
// create dynamic object with 2 fields:
val accessor = dynamic {
get { name ->
when(name) {
// constant field
"foo" -> "bar"
// mutable field
"bar" -> storedValueForBar
else -> throw SymbolNotFoundException()
}
}
set { name, value ->
// only 'bar' is mutable:
if( name == "bar" )
storedValueForBar = value
// the rest is immotable. consider throw also
// SymbolNotFoundException when needed.
else throw IllegalAssignmentException("Can't assign "+name)
}
}
assertEquals("bar", accessor.foo)
assertEquals(null, accessor.bar)
accessor.bar = "buzz"
assertEquals("buzz", accessor.bar)
assertThrows {
accessor.bad = "!23"
}
void
>>> void
Of course, you can return any object from dynamic fields; returning lambdas let create _dynamic methods_ - the callable method. It is very convenient to implement libraries with dynamic remote interfaces, etc.
### Dynamic indexers
Index access for dynamics is passed to the same getter and setter, so it is
generally the same:
var storedValue = "bar"
val x = dynamic {
get {
if( it == "foo" ) storedValue
else null
}
}
assertEquals("bar", x["foo"] )
assertEquals("bar", x.foo )
>>> void
And assigning them works the same. You can make it working
mimicking arrays, but remember, it is not Collection so
collection's sugar won't work with it:
var storedValue = "bar"
val x = dynamic {
get {
when(it) {
"size" -> 1
0 -> storedValue
else -> null
}
}
set { index, value ->
if( index == 0 ) storedValue = value
else throw "Illegal index: "+index
}
}
assertEquals("bar", x[0] )
assertEquals(1, x.size )
x[0] = "buzz"
assertThrows { x[1] = 1 }
assertEquals("buzz", storedValue)
assertEquals("buzz", x[0])
>>> void
If you want dynamic to function like an array, create a [feature
request](https://gitea.sergeych.net/SergeychWorks/lyng/issues).
# Theory
## Basic principles:
@@ -310,3 +604,8 @@ Regular methods are called on instances as usual `instance.method()`. The method
TBD
[argument list](declaring_arguments.md)
### Visibility from within closures and instance scopes
When a closure executes within a method, the closure retains the lexical class context of its creation site. This means private/protected members of that class remain accessible where expected (subject to usual visibility rules). Field resolution checks the declaring class and validates access using the preserved `currentClassCtx`.
See also: [Scopes and Closures: resolution and safety](scopes_and_closures.md)
+1 -1
View File
@@ -87,7 +87,7 @@ You can use Char as both ends of the closed range:
Exclusive end char ranges are supported too:
('a'..<'c').toList
>>> ['a', 'b']
>>> [a,b]
# Instance members
+91
View File
@@ -0,0 +1,91 @@
# Regular expressions
In lyng, you create regular expressions using class `Regex` or `String.re` methods:
assert( "\d*".re is Regex )
assert( Regex("\d*") is Regex )
>>> void
We plan to add slash syntax at some point.
To check that some string matches as whole to some regex:
assert( "123".matches("\d{3}".re) )
assert( !"123".matches("\d{4}".re) )
assert( !"1234".matches("\d".re) )
>>> void
To check that _part of the string_ matches some regular expession, use _match operator_ `=~` just like in Ruby, and its
counterpart, _not match_ operator `!~`:
assert( "abc123def" =~ "\d\d\d".re )
assert( "abc" !~ "\d\d\d".re )
>>> void
When you need to find groups, and more detailed match information, use `Regex.find`:
val result = Regex("abc(\d)(\d)(\d)").find( "bad456 good abc123")
assert( result != null )
assertEquals( 12 .. 17, result.range )
assertEquals( "abc123", result[0] )
assertEquals( "1", result[1] )
assertEquals( "2", result[2] )
assertEquals( "3", result[3] )
>>> void
Note that the object `RegexMatch`, returned by [Regex.find], behaves much like in many other languages: it provides the
index range and groups matches as indexes.
Match operator actually also provides `RegexMatch` in `$~` reserved variable (borrowed from Ruby too):
assert( "bad456 good abc123" =~ "abc(\d)(\d)(\d)".re )
assertEquals( 12 .. 17, $~.range )
assertEquals( "abc123", $~[0] )
assertEquals( "1", $~[1] )
assertEquals( "2", $~[2] )
assertEquals( "3", $~[3] )
>>> void
This is often more readable than calling `find`.
Note that `=~` and `!~` operators against strings and regular expressions are commutative, e.g. regular expression and a
string can be either left or right operator, but not both:
assert( "abc" =~ "\wc".re )
assert( "abc" !~ "\w1c".re )
assert( "a\wc".re =~ "abcd" )
assert( "a[a-z]c".re !~ "a2cd" )
>>> void
Also, string indexing is Regex-aware, and works like `Regex.find` (_not findall!_):
assert( "cd" == "abcdef"[ "c.".re ].value )
>>> void
# Regex class reference
| name | description | notes |
|--------------|-------------------------------------|-------|
| matches(str) | true if the whole `str` matches | |
| find(str) | find first match in `str` or null | (1) |
| findAll(str) | find all matches in `str` as [List] | (1) |
(1)
:: See `RegexMatch` class description below
# RegexMatch
| name | description | notes |
|-------|-------------------------------------------|-------|
| range | the [Range] of the match in source string | |
| value | the value that matches | |
| [n] | [List] of group matches | (1) |
(1)
:: the [0] element is always value, [1] is group 1 match of any, etc.
[List]: List.md
[Range]: Range.md
+52
View File
@@ -0,0 +1,52 @@
# RingBuffer
This is a fixed size buffer that allow to store N last elements with _O(1)_ effectiveness (no data shifting).
Here is the sample:
val r = RingBuffer(3)
assert( r is RingBuffer )
assertEquals(0, r.size)
assertEquals(3, r.capacity)
r += 10
assertEquals(1, r.size)
assertEquals(10, r.first)
r += 20
assertEquals(2, r.size)
assertEquals( [10, 20], r.toList() )
r += 30
assertEquals(3, r.size)
assertEquals( [10, 20, 30], r.toList() )
// now first value is lost:
r += 40
assertEquals(3, r.size)
assertEquals( [20, 30, 40], r.toList() )
assertEquals(3, r.capacity)
>>> void
Ring buffer implements [Iterable], so any of its methods are available for `RingBuffer`, e.g. `first`, `last`, `toList`,
`take`, `drop`, `takelast`, `dropLast`, etc.
## Constructor
RinbBuffer(capacity: Int)
## Instance methods
| method | description | remarks |
|-------------|------------------------|---------|
| capacity | max size of the buffer | |
| size | current size | (1) |
| operator += | add new item | (1) |
| add(item) | add new item | (1) |
| iterator() | return iterator | (1) |
(1)
: Ringbuffer is not threadsafe, protect it with a mutex to avoid RC where necessary.
[Iterable]: Iterable.md
View File
Whitespace-only changes.
+50 -1
View File
@@ -112,6 +112,55 @@ arguments list in almost arbitrary ways. For example:
)
>>> void
,
# Annotations
Annotation in Lyng resembles these proposed for Javascript. Annotation is just regular functions that, if used as annotation, are called when defining a function, var, val or class.
## Function annotation
When used without params, annotation calls a function with two arguments: actual function name and callable function body. Function annotation __must return callable for the function__, either what it received as a second argument (most often), or something else. Annotation name convention is upper scaled:
var annotated = false
// this is annotation function:
fun Special(name, body) {
assertEquals("foo", name)
annotated = true
{ body(it) + 100 }
}
@Special
fun foo(value) { value + 1 }
assert(annotated)
assertEquals(111, foo( 10 ))
>>> void
Function annotation can have more args specified at call time. There arguments must follow two mandatory ones (name and body). Use default values in order to allow parameterless annotation to be used simultaneously.
val registered = Map()
// it is recommended to provide defaults for extra parameters:
fun Registered(name, body, overrideName = null) {
registered[ overrideName ?: name ] = body
body
}
// witout parameters is Ok as we provided default value
@Registered
fun foo() { "called foo" }
@Registered("bar")
fun foo2() { "called foo2" }
assertEquals(registered["foo"](), "called foo")
assertEquals(registered["bar"](), "called foo2")
>>> void
[parallelism]: parallelism.md
## Scopes and Closures: resolution and safety
Closures and dynamic scope graphs require care to avoid accidental recursion and to keep name resolution predictable. See the dedicated page for detailed rules, helper APIs, and best practices:
- Scopes and Closures: resolution and safety → [scopes_and_closures.md](scopes_and_closures.md)
-24
View File
@@ -1,24 +0,0 @@
# Classes
## Declaring
class Foo1
class Foo2() // same, empty constructor
class Foo3() { // full
}
class Foo4 { // Only body
}
```
class_declaration = ["abstract",] "class" [, constructor] [, body]
constructor = "(", [field [, field]] ")
field = [visibility ,] [access ,] name [, typedecl]
body = [visibility] ("var", vardecl) | ("val", vardecl) | ("fun", fundecl)
visibility = "private" | "protected" | "internal"
```
### Abstract classes
Contain one pr more abstract methods which must be implemented; though they
can have constructors, the instances of the abstract classes could not be
created independently
+61 -4
View File
@@ -1,11 +1,13 @@
# Declaring arguments in Lyng
[//]: # (topMenu)
It is a common thing that occurs in many places in Lyng, function declarations,
lambdas and class declarations.
## Regular
## default values
## Default values
Default parameters should not be mixed with mandatory ones:
@@ -71,7 +73,7 @@ destructuring arrays when calling functions and lambdas:
[ first, last ]
}
getFirstAndLast( ...(1..10) ) // see "splats" section below
>>> [1, 10]
>>> [1,10]
# Splats
@@ -83,7 +85,7 @@ or whatever implementing [Iterable], is called _splats_. Here is how we use it:
}
val array = [1,2,3]
testSplat("start", ...array, "end")
>>> ["start", 1, 2, 3, "end"]
>>> [start,1,2,3,end]
>>> void
There could be any number of splats at any positions. You can splat any other [Iterable] type:
@@ -93,8 +95,63 @@ There could be any number of splats at any positions. You can splat any other [I
}
val range = 1..3
testSplat("start", ...range, "end")
>>> ["start", 1, 2, 3, "end"]
>>> [start,1,2,3,end]
>>> void
## Named arguments in calls
Lyng supports named arguments at call sites using colon syntax `name: value`:
```lyng
fun test(a="foo", b="bar", c="bazz") { [a, b, c] }
assertEquals(["foo", "b", "bazz"], test(b: "b"))
assertEquals(["a", "bar", "c"], test("a", c: "c"))
```
Rules:
- Named arguments must follow positional arguments. After the first named argument, no positional arguments may appear inside the parentheses.
- The only exception is the syntactic trailing block after the call: `f(args) { ... }`. This block is outside the parentheses and is handled specially (see below).
- A named argument cannot reassign a parameter already set positionally.
- If the last parameter has already been assigned by a named argument (or named splat), a trailing block is not allowed and results in an error.
Why `:` and not `=` at call sites? In Lyng, `=` is an expression (assignment), so we use `:` to avoid ambiguity. Declarations continue to use `:` for types, while call sites use `as` / `as?` for type operations.
## Named splats (map splats)
Splat (`...`) of a Map provides named arguments to the call. Only string keys are allowed:
```lyng
fun test(a="a", b="b", c="c", d="d") { [a, b, c, d] }
val r = test("A?", ...Map("d" => "D!", "b" => "B!"))
assertEquals(["A?","B!","c","D!"], r)
```
The same with a map literal is often more concise. Define the literal, then splat the variable:
fun test(a="a", b="b", c="c", d="d") { [a, b, c, d] }
val patch = { d: "D!", b: "B!" }
val r = test("A?", ...patch)
assertEquals(["A?","B!","c","D!"], r)
>>> void
Constraints:
- Map splat keys must be strings; otherwise, a clean error is thrown.
- Named splats cannot duplicate parameters already assigned (by positional or named arguments).
- Named splats must follow all positional arguments and positional splats.
- Ellipsis parameters (variadic) remain positional-only and cannot be assigned by name.
## Trailing-lambda rule interaction
If a call is immediately followed by a block `{ ... }`, it is treated as an extra last argument and bound to the last parameter. However, if the last parameter is already assigned by a named argument or a named splat, using a trailing block is an error:
```lyng
fun f(x, onDone) { onDone(x) }
f(x: 1) { 42 } // ERROR
f(1) { 42 } // OK
```
[tutorial]: tutorial.md
+28
View File
@@ -0,0 +1,28 @@
[//]: # (excludeFromIndex)
# String
# This document is for developer notes only
--------------------------
## Interpolation proposal
"""no $iterpolation"""
val inpterpolation1 = "foo"
"${interpolation1}"
>>> "foo"
"$interpolation2"
"no $$ interpolatino"
## Regexp vs div / ?
```EBNF
regex_literal = "/", { regchar }, "/", [ flag ]
foag = "i" | "n"....
regchar = x`x`
```
+2
View File
@@ -1,5 +1,7 @@
# Modules inclusion
[//]: # (excludeFromIndex)
Module is, at the low level, a statement that modifies a given context by adding
here local and exported symbols, performing some tasks and even returning some value
we don't need for now.
+15
View File
@@ -0,0 +1,15 @@
[//]: # (excludeFromIndex)
Provide:
fun outer(a1)
// a1 is caller.a1:arg
val a1_local = a1 + 1
// we return lambda:
{ it ->
// a1_local
a1_lcoal + it
}
}
+236
View File
@@ -0,0 +1,236 @@
# Embedding Lyng in your Kotlin project
Lyng is a tiny, embeddable, Kotlin‑first scripting language. This page shows, step by step, how to:
- add Lyng to your build
- create a runtime and execute scripts
- define functions and variables from Kotlin
- read variable values back in Kotlin
- call Lyng functions from Kotlin
- create your own packages and import them in Lyng
All snippets below use idiomatic Kotlin and rely on Lyng public APIs. They work on JVM and other Kotlin Multiplatform targets supported by `lynglib`.
Note: all Lyng APIs shown are `suspend`, because script evaluation is coroutine‑friendly and can suspend.
### 1) Add Lyng to your build
Add the repository where you publish Lyng artifacts and the dependency on the core library `lynglib`.
Gradle Kotlin DSL (build.gradle.kts):
```kotlin
repositories {
// Your standard repos
mavenCentral()
// If you publish to your own Maven (example: Gitea packages). Adjust URL/token as needed.
maven(url = uri("https://gitea.sergeych.net/api/packages/SergeychWorks/maven"))
}
dependencies {
// Multiplatform: place in appropriate source set if needed
implementation("net.sergeych:lynglib:1.0.0-SNAPSHOT")
}
```
If you use Kotlin Multiplatform, add the dependency in the `commonMain` source set (and platform‑specific sets if you need platform APIs).
### 2) Create a runtime (Scope) and execute scripts
The easiest way to get a ready‑to‑use scope with standard packages is via `Script.newScope()`.
```kotlin
fun main() = kotlinx.coroutines.runBlocking {
val scope = Script.newScope() // suspends on first init
// Evaluate a one‑liner
val result = scope.eval("1 + 2 * 3")
println("Lyng result: $result") // ObjReal/ObjInt etc.
}
```
You can also pre‑compile a script and execute it multiple times:
```kotlin
val script = Compiler.compile("""
// any Lyng code
val x = 40 + 2
x
""")
val run1 = script.execute(scope)
val run2 = script.execute(scope)
```
`Scope.eval("...")` is a shortcut that compiles and executes on the given scope.
### 3) Define variables from Kotlin
To expose data to Lyng, add constants (read‑only) or mutable variables to the scope. All values in Lyng are `Obj` instances; the core types live in `net.sergeych.lyng.obj`.
```kotlin
// Read‑only constant
scope.addConst("pi", ObjReal(3.14159))
// Mutable variable: create or update
scope.addOrUpdateItem("counter", ObjInt(0))
// Use it from Lyng
scope.eval("counter = counter + 1")
```
Tip: Lyng values can be converted back to Kotlin with `toKotlin(scope)`:
```kotlin
val current = (scope.eval("counter")).toKotlin(scope) // Any? (e.g., Int/Double/String/List)
```
### 4) Add Kotlin‑backed functions
Use `Scope.addFn`/`addVoidFn` to register functions implemented in Kotlin. Inside the lambda, use `this.args` to access arguments and return an `Obj`.
```kotlin
// A function returning value
scope.addFn<ObjInt>("inc") {
val x = args.firstAndOnly() as ObjInt
ObjInt(x.value + 1)
}
// A void function (returns Lyng Void)
scope.addVoidFn("log") {
val items = args.list // List<Obj>
println(items.joinToString(" ") { it.toString(this).value })
}
// Call them from Lyng
scope.eval("val y = inc(41); log('Answer:', y)")
```
You can register multiple names (aliases) at once: `addFn<ObjInt>("inc", "increment") { ... }`.
### 5) Read variable values back in Kotlin
The simplest approach: evaluate an expression that yields the value and convert it.
```kotlin
val kotlinAnswer = scope.eval("(1 + 2) * 3").toKotlin(scope) // -> 9 (Int)
// After scripts manipulate your vars:
scope.addOrUpdateItem("name", ObjString("Lyng"))
scope.eval("name = name + ' rocks!'")
val kotlinName = scope.eval("name").toKotlin(scope) // -> "Lyng rocks!"
```
Advanced: you can also grab a variable record directly via `scope.get(name)` and work with its `Obj` value, but evaluating `"name"` is often clearer and enforces Lyng semantics consistently.
### 6) Execute scripts with parameters; call Lyng functions from Kotlin
There are two convenient patterns.
1) Evaluate a Lyng call expression directly:
```kotlin
// Suppose Lyng defines: fun add(a, b) = a + b
scope.eval("fun add(a, b) = a + b")
val sum = scope.eval("add(20, 22)").toKotlin(scope) // -> 42
```
2) Call a Lyng function by name via a prepared call scope:
```kotlin
// Ensure the function exists in the scope
scope.eval("fun add(a, b) = a + b")
// Look up the function object
val addFn = scope.get("add")!!.value as Statement
// Create a child scope with arguments (as Lyng Objs)
val callScope = scope.createChildScope(
args = Arguments(listOf(ObjInt(20), ObjInt(22)))
)
val resultObj = addFn.execute(callScope)
val result = resultObj.toKotlin(scope) // -> 42
```
If you need to pass complex data (lists, maps), construct the corresponding Lyng `Obj` types (`ObjList`, `ObjMap`, etc.) and pass them in `Arguments`.
### 7) Create your own packages and import them in Lyng
Lyng supports packages that are imported from scripts. You can register packages programmatically via `ImportManager` or by providing source texts that declare `package ...`.
Key concepts:
- `ImportManager` holds package registrations and lazily builds `ModuleScope`s when first imported.
- Every `Scope` has `currentImportProvider` and (if it’s an `ImportManager`) a convenience `importManager` to register packages.
Register a Kotlin‑built package:
```kotlin
val scope = Script.newScope()
// Access the import manager behind this scope
val im: ImportManager = scope.importManager
// Register a package "my.tools"
im.addPackage("my.tools") { module: ModuleScope ->
// Expose symbols inside the module scope
module.addConst("version", ObjString("1.0"))
module.addFn<ObjInt>("triple") {
val x = args.firstAndOnly() as ObjInt
ObjInt(x.value * 3)
}
}
// Use it from Lyng
scope.eval("""
import my.tools.*
val v = triple(14)
""")
val v = scope.eval("v").toKotlin(scope) // -> 42
```
Register a package from Lyng source text:
```kotlin
val pkgText = """
package math.extra
fun sqr(x) = x * x
""".trimIndent()
scope.importManager.addTextPackages(pkgText)
scope.eval("""
import math.extra.*
val s = sqr(12)
""")
val s = scope.eval("s").toKotlin(scope) // -> 144
```
You can also register from parsed `Source` instances via `addSourcePackages(source)`.
### 8) Executing from files, security, and isolation
- To run code from a file, read it and pass to `scope.eval(text)` or compile with `Compiler.compile(Source(fileName, text))`.
- `ImportManager` takes an optional `SecurityManager` if you need to restrict what packages or operations are available. By default, `Script.defaultImportManager` allows everything suitable for embedded use; clamp it down in sandboxed environments.
- For isolation, create fresh modules/scopes via `Scope.new()` or `Script.newScope()` when you need a clean environment per request.
```kotlin
// Fresh module based on the default manager, without the standard prelude
val isolated = net.sergeych.lyng.Scope.new()
```
### 9) Tips and troubleshooting
- All values that cross the boundary must be Lyng `Obj` instances. Convert Kotlin values explicitly (e.g., `ObjInt`, `ObjReal`, `ObjString`).
- Use `toKotlin(scope)` to get Kotlin values back. Collections convert to Kotlin collections recursively.
- Most public API in Lyng is suspendable. If you are not already in a coroutine, wrap calls in `runBlocking { ... }` on the JVM for quick tests.
- When registering packages, names must be unique. Register before you compile/evaluate scripts that import them.
- To debug scope content, `scope.toString()` and `scope.trace()` can help during development.
---
That’s it. You now have Lyng embedded in your Kotlin app: you can expose your app’s API, evaluate user scripts, and organize your own packages to import from Lyng code.
+42 -1
View File
@@ -122,6 +122,32 @@ This way, in turn, can also be shortened, as it is overly popular:
The trick, though, works with strings only, and always provide `Exception` instances, which is good for debugging but
most often not enough.
# Exception class
Serializable class that conveys information about the exception. Important members and methods are:
| name | description |
|-------------------|--------------------------------------------------------|
| message | String message |
| stackTrace | lyng stack trace, list of `StackTraceEntry`, see below |
| printStackTrace() | format and print stack trace using println() |
## StackTraceEntry
A simple structire that stores single entry in Lyng stack, it is created automatically on exception creation:
```kotlin
class StackTraceEntry(
val sourceName: String,
val line: Int,
val column: Int,
val sourceString: String
)
```
- `sourceString` is a line extracted from sources. Note that it _is serialized and printed_, so if you want to conceal it, catch all exceptions and filter out sensitive information.
# Custom error classes
_this functionality is not yet released_
@@ -131,7 +157,7 @@ _this functionality is not yet released_
| class | notes |
|----------------------------|-------------------------------------------------------|
| Exception | root of al throwable objects |
| NullReferenceException | |
| NullReferenceException | |
| AssertionFailedException | |
| ClassCastException | |
| IndexOutOfBoundsException | |
@@ -143,3 +169,18 @@ _this functionality is not yet released_
| UnknownException | unexpected kotlin exception caught |
| | |
### Symbol resolution errors
For compatibility, `SymbolNotFound` is an alias of `SymbolNotDefinedException`. You can catch either name in examples and tests.
Example:
```lyng
try {
nonExistingMethod()
}
catch(e: SymbolNotFound) {
// handle
}
```
+34
View File
@@ -0,0 +1,34 @@
## Fix: prevent cycles in scope parent chain during pooled frame reuse
### What changed
- Scope.resetForReuse now fully detaches a reused scope from its previous chain/state before re-parenting:
- sets `parent = null` and regenerates `frameId`
- clears locals/slots/bindings caches
- only after that, validates the new parent with `ensureNoCycle` and assigns it
- ScopePool.borrow on all targets (JVM, Android, JS, Native, Wasm) now has a defensive fallback:
- if `resetForReuse` throws `IllegalStateException` indicating a parent-chain cycle, the pool allocates a fresh `Scope` instead of failing.
### Why
In some nested call patterns (notably instance method calls where an instance is produced by another function and immediately used), the same pooled `Scope` object can be rebound into a chain that already (transitively) contains it. Reassigning `parent` in that case forms a structural cycle, which `ensureNoCycle` correctly detects and throws. This could surface as:
```
IllegalStateException: cycle detected in scope parent chain assignment
at net.sergeych.lyng.Scope.ensureNoCycle(...)
at net.sergeych.lyng.Scope.resetForReuse(...)
at net.sergeych.lyng.ScopePool.borrow(...)
... during instance method invocation
```
The fix removes the failure mode by:
1) Detaching the reused frame from its prior chain/state before validating and assigning the new parent.
2) Falling back to a new frame allocation if a cycle is still detected (extremely rare and cheap vs. a crash).
### Expected effects
- Eliminates sporadic `cycle detected in scope parent chain` crashes during instance method invocation.
- No change to public API or normal semantics.
- Pooling remains enabled by default; the fallback only triggers on the detected cycle edge case.
- Negligible performance impact: fresh allocation is used only when a cycle would have crashed the VM previously.
### Notes
- The fix is platform-wide (all ScopePool actuals are covered).
- We recommend adding/keeping a regression test that exercises: a class with a method, a function returning an instance, and an exported function calling the instance method. The test should pass without exceptions.
+62
View File
@@ -0,0 +1,62 @@
# Lyng formatter (core, CLI, and IDE)
This document describes the Lyng code formatter included in this repository. The formatter lives in the core library (`:lynglib`), is available from the CLI (`lyng fmt`), and is used by the IntelliJ plugin.
## Core library
Package: `net.sergeych.lyng.format`
- `LyngFormatConfig`
- `indentSize` (default 4)
- `useTabs` (default false)
- `continuationIndentSize` (default 8)
- `maxLineLength` (default 120)
- `applySpacing` (default false)
- `applyWrapping` (default false)
- `LyngFormatter`
- `reindent(text, config)` — recomputes indentation from scratch (braces, `else/catch/finally` alignment, continuation indent under `(` `)` and `[` `]`), idempotent.
- `format(text, config)` — runs `reindent` and, depending on `config`, optionally applies:
- a safe spacing pass (commas/operators/colons/keyword parens; member access `.` remains tight; no changes to strings/comments), and
- a controlled wrapping pass for long call arguments (no trailing commas).
Both passes are designed to be idempotent. Extensive tests live under `:lynglib/src/commonTest/.../format`.
## CLI formatter
```
lyng fmt [--check] [--in-place|-i] [--spacing] [--wrap] <file1.lyng> [file2.lyng ...]
```
- Defaults: indent-only; spacing and wrapping are OFF unless flags are provided.
- `--check` prints files that would change and exits with code 2 if any changes are detected.
- `--in-place`/`-i` rewrites files in place (default if not using `--check`).
- `--spacing` enables the safe spacing pass (commas/operators/colons/keyword parens).
- `--wrap` enables controlled wrapping of long call argument lists (respects `maxLineLength`, no trailing commas).
Examples:
```
# check formatting without modifying files
lyng fmt --check docs/samples/fs_sample.lyng
# format in place with spacing rules enabled
lyng fmt --spacing -i docs/samples/fs_sample.lyng
# format in place with spacing + wrapping
lyng fmt --spacing --wrap -i src/**/*.lyng
```
## IntelliJ plugin
- Indentation: always enabled, idempotent; the plugin computes per-line indent via the core formatter.
- Spacing/wrapping: optional and OFF by default.
- Settings/Preferences → Lyng Formatter provides toggles:
- "Enable spacing normalization (commas/operators/colons/keyword parens)"
- "Enable line wrapping (120 cols) [experimental]"
- Reformat Code applies: indentation first, then spacing, then wrapping if toggled.
## Design notes
- Single source of truth: The core formatter is used by CLI and IDE to keep behavior consistent.
- Stability first: Spacing/wrapping are gated by flags/toggles; indentation from scratch is always safe and idempotent.
- Non-destructive: The formatter carefully avoids changing string/char literals and comment contents.
+29
View File
@@ -0,0 +1,29 @@
# Plugin for IntelliJ IDEA
[//]: # (excludeFromIndex)
We introduce the alpha version of the plugin for IntelliJ IDEA 2024.3.x+ IDE variants. It is compatible with 2025.x and
should be compatible with other IDEA flavors, notably [OpenIDE](https://openide.ru/). It supports the following features:
- syntax highlighting (2 stage, fast and more accurate that analyses in background)
- reformat code (indents, spaces)
- reformat on paste
- smart enter key
Features are configurable via the plugin settings page, in system settings.
> Recommended for IntelliJ-based IDEs: While IntelliJ can import TextMate bundles
> (Settings/Preferences → Editor → TextMate Bundles), the native Lyng plugin provides
> better support (formatting, smart enter, background analysis, etc.). Prefer installing
> this plugin over using a TextMate bundle.
### Install
- From ZIP: download the archive below, then in IntelliJ IDEA open Settings/Preferences → Plugins →
gear icon → Install Plugin from Disk… and select the downloaded ZIP. Restart IDE if prompted.
- Alternatively, if/when the plugin is published to a marketplace, you will be able to install it
directly from the “Marketplace” tab (not yet available).
### [Download plugin v0.0.2-SNAPSHOT](https://lynglang.com/distributables/lyng-idea-0.0.2-SNAPSHOT.zip)
Your ideas and bugreports are welcome on the [project gitea page](https://gitea.sergeych.net/SergeychWorks/lyng/issues)
+167
View File
@@ -0,0 +1,167 @@
# Json support
Since 1.0.5 we start adding JSON support. Versions 1,0,6* support serialization of the basic types, including lists and
maps, and simple classes. Multiple inheritance may produce incorrect results, it is work in progress.
## Serialization in Lyng
// in lyng
assertEquals("{\"a\":1}", {a: 1}.toJsonString())
void
>>> void
Simple classes serialization is supported:
import lyng.serialization
class Point(foo,bar) {
val t = 42
}
// val is not serialized
assertEquals( "{\"foo\":1,\"bar\":2}", Point(1,2).toJsonString() )
>>> void
Note that mutable members are serialized:
import lyng.serialization
class Point2(foo,bar) {
var reason = 42
// but we override json serialization:
fun toJsonObject() {
{ "custom": true }
}
}
// var is serialized instead
assertEquals( "{\"custom\":true}", Point2(1,2).toJsonString() )
>>> void
Custom serialization of user classes is possible by overriding `toJsonObject` method. It must return an object which is
serializable to Json. Most often it is a map, but any object is accepted, that makes it very flexible:
import lyng.serialization
class Point2(foo,bar) {
var reason = 42
// but we override json serialization:
fun toJsonObject() {
{ "custom": true }
}
}
class Custom {
fun toJsonObject() {
"full freedom"
}
}
// var is serialized instead
assertEquals( "\"full freedom\"", Custom().toJsonString() )
>>> void
Please note that `toJsonString` should be used to get serialized string representation of the object. Don't call
`toJsonObject` directly, it is not intended to be used outside the serialization library.
## Kotlin side interfaces
The "Batteries included" principle is also applied to serialization.
- `Obj.toJson()` provides Kotlin `JsonElement`
- `Obj.toJsonString()` provides Json string representation
- `Obj.decodeSerializableWith()` and `Obj.decodeSerializable()` allows to decode Lyng classes as Kotlin objects using
`kotlinx.serialization`:
```kotlin
/**
* Decodes the current object into a deserialized form using the provided deserialization strategy.
* It is based on [Obj.toJson] and uses existing Kotlin Json serialization, without string representation
* (only `JsonElement` to carry information between Kotlin and Lyng serialization worlds), thus efficient.
*
* @param strategy The deserialization strategy that defines how the object should be decoded.
* @param scope An optional scope used during deserialization to define the context. Defaults to a new instance of Scope.
* @return The deserialized object of type T.
*/
suspend fun <T> Obj.decodeSerializableWith(strategy: DeserializationStrategy<T>, scope: Scope = Scope()): T =
Json.decodeFromJsonElement(strategy, toJson(scope))
/**
* Decodes a serializable object of type [T] using the provided decoding scope. The deserialization uses
* [Obj.toJson] and existing Json based serialization ithout using actual string representation, thus
* efficient.
*
* @param T The type of the object to be decoded. Must be a reified type.
* @param scope The scope used during decoding. Defaults to a new instance of [Scope].
*/
suspend inline fun <reified T> Obj.decodeSerializable(scope: Scope = Scope()) =
decodeSerializableWith<T>(serializer<T>(), scope)
```
Note that lyng-2-kotlin deserialization with `kotlinx.serialization` uses JsonElement as information carrier without
formatting and parsing actual Json strings. This is why we use `Json.decodeFromJsonElement` instead of
`Json.decodeFromString`. Such an approach gives satisfactory performance without writing and supporting custom
`kotlinx.serialization` codecs.
### Pitfall: JSON objects and Map<String, Any?>
Kotlin serialization does not support `Map<String, Any?>` as a serializable type, more general, it can't serialize `Any`. This in particular means that you can deserialize Kotlin `Map<String, T>` as long as `T` is `@Serializable` in Kotlin:
```kotlin
@Serializable
data class TestJson2(
val value: Int,
val inner: Map<String,Int>
)
@Test
fun deserializeMapWithJsonTest() = runTest {
val x = eval("""
import lyng.serialization
{ value: 1, inner: { "foo": 1, "bar": 2 }}
""".trimIndent()).decodeSerializable<TestJson2>()
// That works perfectly well:
assertEquals(TestJson2(1, mapOf("foo" to 1, "bar" to 2)), x)
}
```
But what if your map has objects of different types? The approach of using polymorphism is partially applicable, but what to do with `{ one: 1, two: "two" }`?
The answer is pretty simple: use `JsonObject` in your deserializable object. This class is capable of holding any JSON types and structures and is sort of a silver bullet for such cases:
~~~kotlin
@Serializable
data class TestJson3(
val value: Int,
val inner: JsonObject
)
@Test
fun deserializeAnyMapWithJsonTest() = runTest {
val x = eval("""
import lyng.serialization
{ value: 12, inner: { "foo": 1, "bar": "two" }}
""".trimIndent()).decodeSerializable<TestJson3>()
assertEquals(TestJson3(12, JsonObject(mapOf("foo" to JsonPrimitive(1), "bar" to Json.encodeToJsonElement("two")))), x)
}
~~~
# List of supported types
| Lyng type | JSON type | notes |
|-----------|-----------|-------------|
| `Int` | number | |
| `Real` | number | |
| `String` | string | |
| `Bool` | boolean | |
| `null` | null | |
| `Instant` | string | ISO8601 (1) |
| `List` | array | (2) |
| `Map` | object | (2) |
(1)
: ISO8601 flavor 1970-05-06T06:00:00.000Z in used; number of fractional digits depends on the truncation
on [Instant](time.md), see `Instant.truncateTo...` functions.
(2)
: List may contain any objects serializable to Json.
(3)
: Map keys must be strings, map values may be any objects serializable to Json.
+228
View File
@@ -0,0 +1,228 @@
### lyng.io.fs — async filesystem access for Lyng scripts
This module provides a uniform, suspend-first filesystem API to Lyng scripts, backed by Kotlin Multiplatform implementations.
- JVM/Android/Native: Okio `FileSystem.SYSTEM` (non-blocking via coroutine dispatcher)
- JS/Node: Node filesystem (currently via Okio node backend; a native `fs/promises` backend is planned)
- JS/Browser and Wasm: in-memory virtual filesystem for now
It exposes a Lyng class `Path` with methods for file and directory operations, including streaming readers for large files.
It is a separate library because access to teh filesystem is a security risk we compensate with a separate API that user must explicitly include to the dependency and allow. Together with `FsAceessPolicy` that is required to `createFs()` which actually adds the filesystem to the scope, the security risk is isolated.
Also, it helps keep Lyng core small and focused.
---
#### Add the library to your project (Gradle)
If you use this repository as a multi-module project, add a dependency on `:lyngio`:
```kotlin
dependencies {
implementation("net.sergeych:lyngio:0.0.1-SNAPSHOT")
}
```
Note on maven repository. Lyngio uses ths same maven as Lyng code (`lynglib`) so it is most likely already in your project. If ont, add it to the proper section of your `build.gradle.kts` or settings.gradle.kts:
```kotlin
repositories {
maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
}
```
This brings in:
- `:lynglib` (Lyng engine)
- Okio (`okio`, `okio-fakefilesystem`, and `okio-nodefilesystem` for JS)
- Kotlin coroutines
---
#### Install the module into a Lyng Scope
The filesystem module is not installed automatically. You must explicitly register it in the scope’s `ImportManager` using the installer. You can customize access control via `FsAccessPolicy`.
Kotlin (host) bootstrap example (imports omitted for brevity):
```kotlin
val scope: Scope = Scope.new()
val installed: Boolean = createFs(PermitAllAccessPolicy, scope)
// installed == true on first registration in this ImportManager, false on repeats
// In scripts (or via scope.eval), import the module to use its symbols:
scope.eval("import lyng.io.fs")
```
You can install with a custom policy too (see Access policy below).
---
#### Using from Lyng scripts
```lyng
val p = Path("/tmp/hello.txt")
// Text I/O
p.writeUtf8("Hello Lyng!\n")
println(p.readUtf8())
// Binary I/O
val data = Buffer.fromHex("deadbeef")
p.writeBytes(data)
// Existence and directories
assertTrue(p.exists())
Path("/tmp/work").mkdirs()
// Listing
for (entry in Path("/tmp").list()) {
println(entry)
}
// Globbing
val txts = Path("/tmp").glob("**/*.txt").toList()
// Copy / Move / Delete
Path("/tmp/a.txt").copy("/tmp/b.txt", overwrite=true)
Path("/tmp/b.txt").move("/tmp/c.txt", overwrite=true)
Path("/tmp/c.txt").delete()
// Streaming large files (does not load whole file into memory)
var bytes = 0
val it = Path("/tmp/big.bin").readChunks(1_048_576) // 1MB chunks
val iter = it.iterator()
while (iter.hasNext()) {
val chunk = iter.next()
bytes = bytes + chunk.size()
}
// Text chunks and lines
for (s in Path("/tmp/big.txt").readUtf8Chunks(64_000)) {
// process each string chunk
}
for (ln in Path("/tmp/big.txt").lines()) {
// process line by line
}
```
---
#### API (Lyng class `Path`)
Constructor:
- `Path(path: String)` — creates a Path object
- `Paths(path: String)` — alias
File and directory operations (all suspend under the hood):
- `name`: name, `String`
- `segments`: list of parsed path segments (directories)
- `parent`: parent directory, `Path?`; null if root
- `exists(): Bool`
- `isFile(): Bool` — true if the path points to a regular file (cached metadata)
- `isDirectory(): Bool` — true if the path points to a directory (cached metadata)
- `size(): Int?` — size in bytes or null if unknown (cached metadata)
- `createdAt(): Instant?` — creation time as Lyng `Instant`, or null (cached metadata)
- `createdAtMillis(): Int?` — creation time in epoch milliseconds, or null (cached metadata)
- `modifiedAt(): Instant?` — last modification time as Lyng `Instant`, or null (cached metadata)
- `modifiedAtMillis(): Int?` — last modification time in epoch milliseconds, or null (cached metadata)
- `list(): List<Path>` — children of a directory
- `readBytes(): Buffer`
- `writeBytes(bytes: Buffer)`
- `appendBytes(bytes: Buffer)`
- `readUtf8(): String`
- `writeUtf8(text: String)`
- `appendUtf8(text: String)`
- `metadata(): Map` — keys: `isFile`, `isDirectory`, `size`, `createdAtMillis`, `modifiedAtMillis`, `isSymlink`
- `mkdirs(mustCreate: Bool = false)`
- `move(to: Path|String, overwrite: Bool = false)`
- `delete(mustExist: Bool = false, recursively: Bool = false)`
- `copy(to: Path|String, overwrite: Bool = false)`
- `glob(pattern: String): List<Path>` — supports `**`, `*`, `?` (POSIX-style)
Streaming readers for big files:
- `readChunks(size: Int = 65536): Iterator<Buffer>` — iterate fixed-size byte chunks
- `readUtf8Chunks(size: Int = 65536): Iterator<String>` — iterate text chunks by character count
- `lines(): Iterator<String>` — line iterator built on `readUtf8Chunks`
Notes:
- Iterators implement Lyng iterator protocol. If you break early from a loop, the runtime will attempt to call `cancelIteration()` when available.
- Current implementations chunk in memory. The public API is stable; internals will evolve to true streaming on all platforms.
- Attribute accessors (`isFile`, `isDirectory`, `size`, `createdAt*`, `modifiedAt*`) cache a metadata snapshot inside the `Path` instance to avoid repeated filesystem calls during a sequence of queries. `metadata()` remains available for bulk access.
---
#### Access policy (security)
Access control is enforced by `FsAccessPolicy`. You pass a policy at installation time. The module wraps the filesystem with a secured decorator that consults the policy for each primitive operation.
Main types:
- `FsAccessPolicy` — your policy implementation
- `PermitAllAccessPolicy` — allows all operations (default for testing)
- `AccessOp` (sealed) — operations the policy can decide on:
- `ListDir(path)`
- `CreateFile(path)`
- `OpenRead(path)`
- `OpenWrite(path)`
- `OpenAppend(path)`
- `Delete(path)`
- `Rename(from, to)`
- `UpdateAttributes(path)` — defaults to write-level semantics
Minimal denying policy example (imports omitted for brevity):
```kotlin
val denyWrites = object : FsAccessPolicy {
override suspend fun check(op: AccessOp, ctx: AccessContext): AccessDecision = when (op) {
is AccessOp.OpenRead, is AccessOp.ListDir -> AccessDecision(Decision.Allow)
else -> AccessDecision(Decision.Deny, reason = "read-only policy")
}
}
createFs(denyWrites, scope)
scope.eval("import lyng.io.fs")
```
Composite operations like `copy` and `move` are checked as a set of primitives (e.g., `OpenRead(src)` + `Delete(dst)` if overwriting + `CreateFile(dst)` + `OpenWrite(dst)`).
---
#### Errors and exceptions
Policy denials are surfaced as Lyng runtime errors, not raw Kotlin exceptions:
- Internally, a denial throws `AccessDeniedException`. The module maps it to `ObjIllegalOperationException` wrapped into an `ExecutionError` visible to scripts.
Examples (Lyng):
```lyng
import lyng.io.fs
val p = Path("/protected/file.txt")
try {
p.writeUtf8("x")
fail("expected error")
} catch (e) {
// e is an ExecutionError; message contains the policy reason
}
```
Other I/O failures (e.g., not found, not a directory) are also raised as Lyng errors (`ObjIllegalStateException`, `ObjIllegalArgumentException`, etc.) depending on context.
---
#### Platform notes
- JVM/Android/Native: synchronous Okio calls are executed on `Dispatchers.IO` (JVM/Android) or `Dispatchers.Default` (Native) to avoid blocking the main thread.
- NodeJS: currently uses Okio’s Node backend. For heavy I/O, a native `fs/promises` backend is planned to fully avoid event-loop blocking.
- Browser/Wasm: uses an in-memory filesystem for now. Persistent backends (IndexedDB or File System Access API) are planned.
---
#### Roadmap
- Native NodeJS backend using `fs/promises`
- Browser persistent storage (IndexedDB)
- Streaming readers/writers over real OS streams
- Attribute setters and richer metadata
If you have specific needs (e.g., sandboxing, virtual roots), implement a custom `FsAccessPolicy` or ask us to add a helper.
+141
View File
@@ -0,0 +1,141 @@
### Lyng CLI (`lyng`)
The Lyng CLI is the reference command-line tool for the Lyng language. It lets you:
- Run Lyng scripts from files or inline strings (shebangs accepted)
- Use standard argument passing (`ARGV`) to your scripts.
- Format Lyng source files via the built-in `fmt` subcommand.
#### Building on Linux
Requirements:
- JDK 17+ (for Gradle and the JVM distribution)
- GNU zip utilities (for packaging the native executable)
- upx tool (executable in-place compression)
The repository provides convenience scripts in `bin/` for local builds and installation into `~/bin`.
Note: In this repository the scripts are named `bin/local_release` and `bin/local_jrelease`. In some environments these may be aliased as `bin/release` and `bin/jrelease`. The steps below use the actual file names present here.
##### Option A: Native linuxX64 executable (`lyng`)
1) Build the native binary:
```
./gradlew :lyng:linkReleaseExecutableLinuxX64
```
2) Install and package locally:
```
bin/local_release
```
What this does:
- Copies the built executable to `~/bin/lyng` for easy use in your shell.
- Produces `distributables/lyng-linuxX64.zip` containing the `lyng` executable.
##### Option B: JVM distribution (`jlyng` launcher)
This creates a JVM distribution with a launcher script and links it to `~/bin/jlyng`.
```
bin/local_jrelease
```
What this does:
- Runs `./gradlew :lyng:installJvmDist` to build the JVM app distribution to `lyng/build/install/lyng-jvm`.
- Copies the distribution under `~/bin/jlyng-jvm`.
- Creates a symlink `~/bin/jlyng` pointing to the launcher script.
#### Usage
Once installed, ensure `~/bin` is on your `PATH`. You can then use either the native `lyng` or the JVM `jlyng` launcher (both have the same CLI surface).
##### Running scripts
- Run a script by file name and pass arguments to `ARGV`:
```
lyng path/to/script.lyng arg1 arg2
```
- Run a script whose name starts with `-` using `--` to stop option parsing:
```
lyng -- -my-script.lyng arg1 arg2
```
- Execute inline code with `-x/--execute` and pass positional args to `ARGV`:
```
lyng -x "println(\"Hello\")" more args
```
- Print version/help:
```
lyng --version
lyng --help
```
### Use in shell scripts
Standard unix shebangs (`#!`) are supported, so you can make Lyng scripts directly executable on Unix-like systems. For example:
#!/usr/bin/env lyng
println("Hello, world!")
##### Formatting source: `fmt` subcommand
Format Lyng files with the built-in formatter.
Basic usage:
```
lyng fmt [OPTIONS] FILE...
```
Options:
- `--check` — Check-only mode. Prints file paths that would change and exits with code 2 if any changes are needed, 0 otherwise.
- `-i, --in-place` — Write formatted content back to the source files (off by default).
- `--spacing` — Apply spacing normalization.
- `--wrap`, `--wrapping` — Enable line wrapping.
Semantics and exit codes:
- Default behavior is to write formatted content to stdout. When multiple files are provided, the output is separated with `--- <path> ---` headers.
- `--check` and `--in-place` are mutually exclusive; using both results in an error and exit code 1.
- `--check` exits with 2 if any file would change, with 0 otherwise.
- Other errors (e.g., I/O issues) result in a non-zero exit code.
Examples:
```
# Print formatted content to stdout
lyng fmt src/file.lyng
# Format multiple files to stdout with headers
lyng fmt src/a.lyng src/b.lyng
# Check mode: list files that would change; exit 2 if changes are needed
lyng fmt --check src/**/*.lyng
# In-place formatting
lyng fmt -i src/**/*.lyng
# Enable spacing normalization and wrapping
lyng fmt --spacing --wrap src/file.lyng
```
#### Notes
- Both native and JVM distributions expose the same CLI interface. Use whichever best fits your environment.
- When executing scripts, all positional arguments after the script name are available in Lyng as `ARGV`.
- The interpreter recognizes shebang lines (`#!`) at the beginning of a script file and ignores them at runtime, so you can make Lyng scripts directly executable on Unix-like systems.
+25 -5
View File
@@ -9,17 +9,37 @@ Same as in C++.
| **Highest**<br>0 | power, not, calls, indexing, dot,... |
| 1 | `%` `*` `/` |
| 2 | `+` `-` |
| 3 | bit shifts (NI) |
| 3 | bit shifts `<<` `>>` |
| 4 | `<=>` (1) |
| 5 | `<=` `>=` `<` `>` |
| 6 | `==` `!=` |
| 7 | bitwise and `&` (NI) |
| 9 | bitwise or `\|` (NI) |
| 7 | bitwise and `&` |
| 8 | bitwise xor `^` |
| 9 | bitwise or `\|` |
| 10 | `&&` |
| 11<br/>lowest | `\|\|` |
(NI)
: not yet implemented.
Bitwise operators
: available only for `Int` values. For mixed `Int`/`Real` numeric expressions, bitwise operators are not defined.
Bitwise NOT `~x`
: unary operator that inverts all bits of a 64‑bit signed integer (`Int`). It follows two's‑complement rules, so
`~x` is numerically equal to `-(x + 1)`. Examples: `~0 == -1`, `~1 == -2`, `~(-1) == 0`.
Examples:
```
5 & 3 // -> 1
5 | 3 // -> 7
5 ^ 3 // -> 6
~0 // -> -1
1 << 3 // -> 8
8 >> 3 // -> 1
```
Notes:
- Shifts operate on 64-bit signed integers (`Int` is 64-bit). Right shift `>>` is arithmetic (sign-propagating).
- Shift count is masked to the range 0..63, similar to the JVM/Kotlin behavior (e.g., `1 << 65` equals `1 << 1`).
(1)
: Shuttle operator: `a <=> b` returns 0 if a == b, negative Int if a < b and positive Int otherwise. It is necessary to
+117 -5
View File
@@ -1,4 +1,6 @@
# Parallelism in Lyng
# Multithreading/parallel execution
[//]: # (topMenu)
Lyng is built to me multithreaded where possible (e.g. all targets byt JS and wasmJS as for now)
and cooperatively parallel (coroutine based) everywhere.
@@ -36,19 +38,19 @@ Launch has the only argument which should be a callable (lambda usually) that is
## Synchronization: Mutex
Suppose we have a resource, that could be used concurrently, a coutner in our case. If we won'r protect it, concurrent usage cause RC, Race Condition, providing wrong result:
Suppose we have a resource, that could be used concurrently, a counter in our case. If we won't protect it, concurrent usage cause RC, Race Condition, providing wrong result:
var counter = 0
(1..4).map {
(1..50).map {
launch {
// slow increment:
val c = counter
delay(10)
delay(100)
counter = c + 1
}
}.forEach { it.await() }
assert(counter < 4)
assert(counter < 50) { "counter is "+counter }
>>> void
The obviously wrong result is not 4, as all coroutines capture the counter value, which is 1, then sleep for 5ms, then save 1 + 1 as result. May some coroutines will pass, so it will be 1 or 2, most likely.
@@ -128,3 +130,113 @@ Usage example:
yield()
} while(true)
}
# Data exchange for coroutines
## Flow
Flow is an async cold sequence; it is named after kotlin's Flow as it resembles it closely. The cold means the flow is only evaluated when iterated (collected, in Kotlin terms), before it is inactive. Sequence means that it is potentially unlimited, as in our example of glorious Fibonacci number generator:
// Fibonacch numbers flow!
val f = flow {
println("Starting generator")
var n1 = 0
var n2 = 1
emit(n1)
emit(n2)
while(true) {
val n = n1 + n2
emit(n)
n1 = n2
n2 = n
}
}
val correctFibs = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765]
println("Generation starts")
assertEquals( correctFibs, f.take(correctFibs.size))
>>> Generation starts
>>> Starting generator
>>> void
Great: the generator is not executed until collected bu the `f.take()` call, which picks specified number of elements from the flow, can cancel it.
Important difference from the channels or like, every time you collect the flow, you collect it anew:
var isStarted = false
val f = flow {
emit("start")
isStarted = true
(1..4).forEach { emit(it) }
}
// flow is not yet started, e.g. not got execited,
// that is called 'cold':
assertEquals( false, isStarted )
// let's collect flow:
val result = []
for( x in f ) result += x
println(result)
assertEquals( true, isStarted)
// let's collect it once again, it should be the same:
println(f.toList())
// and again:
assertEquals( result, f.toList() )
>>> [start,1,2,3,4]
>>> [start,1,2,3,4]
>>> void
Notice that flow's lambda is not called until actual collection is started. Cold flows are
better in terms of resource consumption.
Flows allow easy transforming of any [Iterable]. See how the standard Lyng library functions use it:
fun Iterable.filter(predicate) {
val list = this
flow {
for( item in list ) {
if( predicate(item) ) {
emit(item)
}
}
}
}
[Iterable]: Iterable.md
## Scope frame pooling (JVM)
Lyng includes an optional optimization for function/method calls on JVM: scope frame pooling, toggled by the runtime flag `PerfFlags.SCOPE_POOL`.
- Default: `SCOPE_POOL` is OFF on JVM.
- Rationale: the current `ScopePool` implementation is not thread‑safe. Lyng targets multi‑threaded execution on most platforms, therefore we keep pooling disabled by default until a thread‑safe design is introduced.
- When safe to enable: single‑threaded runs (e.g., micro‑benchmarks or scripts executed on a single worker) where no scopes are shared across threads.
- How to toggle at runtime (Kotlin/JVM tests):
- `PerfFlags.SCOPE_POOL = true` to enable.
- `PerfFlags.SCOPE_POOL = false` to disable.
- Expected effect (from our JVM micro‑benchmarks): in deep call loops, enabling pooling reduced total time by about 1.38× in a dedicated pooling benchmark; mileage may vary depending on workload.
Future work: introduce thread‑safe pooling (e.g., per‑thread pools or confinement strategies) before considering enabling it by default in multi‑threaded environments.
### Closures inside coroutine helpers (launch/flow)
Closures executed by `launch { ... }` and `flow { ... }` resolve names using the `ClosureScope` rules:
1. Closure frame locals/arguments
2. Captured receiver instance/class members
3. Closure ancestry locals + each frame’s `this` members (cycle‑safe)
4. Caller `this` members
5. Caller ancestry locals + each frame’s `this` members (cycle‑safe)
6. Module pseudo‑symbols (e.g., `__PACKAGE__`)
7. Direct module/global fallback (nearest `ModuleScope` and its parent/root)
Implications:
- Outer locals (e.g., `counter`) stay visible across suspension points.
- Global helpers like `delay(ms)` and `yield()` are available from inside closures.
- If you write your own async helpers, execute user lambdas under `ClosureScope(callScope, capturedCreatorScope)` and avoid manual ancestry walking.
See also: [Scopes and Closures: resolution and safety](scopes_and_closures.md)
+622
View File
@@ -0,0 +1,622 @@
This document explains how to enable and measure the performance optimizations added to the Lyng interpreter. The focus is JVM‑first with safe, flag‑guarded rollouts and quick A/B testing. Other targets (JS/Wasm/Native) keep conservative defaults until validated.
[//]: # (excludeFromIndex)
## Overview
Optimizations are controlled by runtime‑mutable flags in `net.sergeych.lyng.PerfFlags`, initialized from platform‑specific static defaults `net.sergeych.lyng.PerfDefaults` (KMP `expect/actual`).
- JVM/Android defaults are aggressive (e.g. `RVAL_FASTPATH=true`).
- Non‑JVM defaults are conservative (e.g. `RVAL_FASTPATH=false`).
All flags are `var` and can be flipped at runtime (e.g., from tests or host apps) for A/B comparisons.
### Workload presets (JVM‑first)
To simplify switching between recommended flag sets for different workloads, use `net.sergeych.lyng.PerfProfiles`:
```
val snap = PerfProfiles.apply(PerfProfiles.Preset.BENCH) // or BASELINE / BOOKS
// ... run workload ...
PerfProfiles.restore(snap) // restore previous flags
```
- BASELINE: restores platform defaults from `PerfDefaults` (good rollback point).
- BENCH: expression‑heavy micro‑bench focus (aggressive R‑value and PIC optimizations on JVM).
- BOOKS: documentation workloads (prefers simpler paths; disables some PIC/arg builder features shown neutral/negative for this load in A/B).
## Key flags
- `LOCAL_SLOT_PIC` — Runtime cache in `LocalVarRef` to avoid repeated name→slot lookups per frame (ON JVM default).
- `EMIT_FAST_LOCAL_REFS` — Compiler emits `FastLocalVarRef` for identifiers known to be locals/params (ON JVM default).
- `ARG_BUILDER` — Efficient argument building: small‑arity no‑alloc and pooled builder on JVM (ON JVM default).
- `ARG_SMALL_ARITY_12` — Extends small‑arity no‑alloc call paths from 0–8 to 0–12 arguments (JVM‑first exploration; OFF by default). Use for codebases with many 9–12 arg calls; A/B before enabling.
- `SKIP_ARGS_ON_NULL_RECEIVER` — Early return on optional‑null receivers before building args (semantics‑compatible). A/B only.
- `SCOPE_POOL` — Scope frame pooling for calls (JVM, per‑thread ThreadLocal pool). ON by default on JVM; togglable at runtime.
- `FIELD_PIC` — 2‑entry polymorphic inline cache for field reads/writes keyed by `(classId, layoutVersion)` (ON JVM default).
- `METHOD_PIC` — 2‑entry PIC for instance method calls keyed by `(classId, layoutVersion)` (ON JVM default).
- `FIELD_PIC_SIZE_4` — Increases Field PIC size from 2 to 4 entries (JVM-first tuning; OFF by default). Use for sites with >2 receiver shapes.
- `METHOD_PIC_SIZE_4` — Increases Method PIC size from 2 to 4 entries (JVM-first tuning; OFF by default).
- `PIC_ADAPTIVE_2_TO_4` — Adaptive growth of Field/Method PICs from 2→4 entries per-site when miss rate >20% over ≥256 accesses (JVM-first; OFF by default).
- `INDEX_PIC` — Enables polymorphic inline cache for indexing (e.g., `a[i]`) and related fast paths. Defaults to follow `FIELD_PIC` on init; can be toggled independently.
- `INDEX_PIC_SIZE_4` — Increases Index PIC size from 2 to 4 entries (JVM-first tuning). Default: ON for JVM; OFF elsewhere by default.
- `PIC_DEBUG_COUNTERS` — Enable lightweight hit/miss counters via `PerfStats` (OFF by default).
- `PRIMITIVE_FASTOPS` — Fast paths for `(ObjInt, ObjInt)` arithmetic/comparisons and `(ObjBool, ObjBool)` logic (ON JVM default).
- `RVAL_FASTPATH` — Bypass `ObjRecord` in pure expression evaluation via `ObjRef.evalValue` (ON JVM default, OFF elsewhere).
See `src/commonMain/kotlin/net/sergeych/lyng/PerfFlags.kt` and `PerfDefaults.*.kt` for details and platform defaults.
## Where optimizations apply
- Locals: `FastLocalVarRef`, `LocalVarRef` per‑frame cache (PIC).
- Calls: small‑arity zero‑alloc paths (0–8 args; optionally 0–12 with `ARG_SMALL_ARITY_12`), pooled builder (JVM), and child frame pooling (optional).
- Properties/methods: Field/Method PICs with receiver shape `(classId, layoutVersion)` and handle‑aware caches; configurable 2→4 entries under flags.
- Expressions: R‑value fast paths in hot nodes (`UnaryOpRef`, `BinaryOpRef`, `ElvisRef`, logical ops, `RangeRef`, `IndexRef` read, `FieldRef` receiver eval, `ListLiteralRef` elements, `CallRef` callee, `MethodCallRef` receiver, assignment RHS).
- Primitives: Direct boolean/int ops where safe.
### Compiler constant folding (conservative)
- The compiler folds a safe subset of literal‑only expressions at compile time to reduce runtime work:
- Integer arithmetic: `+ - * / %` (division/modulo only when divisor ≠ 0).
- Bitwise integer ops: `& ^ | << >>`.
- Comparisons and equality for ints/strings/chars: `== != < <= > >=`.
- Boolean logic for literal booleans: `|| &&` and unary `!`.
- String concatenation of literal strings: `"a" + "b"`.
- Non‑literal operands or side‑effecting constructs are not folded.
- Semantics remain unchanged; tests verify parity.
## Running JVM micro‑benchmarks
Each benchmark prints timings with `[DEBUG_LOG]` and includes correctness assertions to prevent dead‑code elimination.
Run individual tests to avoid multiplatform matrices:
```
./gradlew :lynglib:jvmTest --tests LocalVarBenchmarkTest
./gradlew :lynglib:jvmTest --tests CallBenchmarkTest
./gradlew :lynglib:jvmTest --tests CallMixedArityBenchmarkTest
./gradlew :lynglib:jvmTest --tests CallSplatBenchmarkTest
./gradlew :lynglib:jvmTest --tests PicBenchmarkTest
./gradlew :lynglib:jvmTest --tests PicInvalidationJvmTest
./gradlew :lynglib:jvmTest --tests PicAdaptiveABTest
./gradlew :lynglib:jvmTest --tests ArithmeticBenchmarkTest
./gradlew :lynglib:jvmTest --tests ExpressionBenchmarkTest
./gradlew :lynglib:jvmTest --tests IndexPicABTest
./gradlew :lynglib:jvmTest --tests IndexWritePathABTest
./gradlew :lynglib:jvmTest --tests CallPoolingBenchmarkTest
./gradlew :lynglib:jvmTest --tests MethodPoolingBenchmarkTest
./gradlew :lynglib:jvmTest --tests MixedBenchmarkTest
./gradlew :lynglib:jvmTest --tests DeepPoolingStressJvmTest
```
Typical output (example):
```
[DEBUG_LOG] [BENCH] mixed-arity x200000 [ARG_BUILDER=ON]: 85.7 ms
```
Lower time is better. Run the same bench with a flag OFF vs ON to compare.
### Optional JFR allocation profiling (JVM)
When running end‑to‑end “book” workloads or heavier benches, you can enable JFR to capture allocation and GC details:
```
./gradlew :lynglib:jvmTest --tests BookAllocationProfileTest -Dlyng.jfr=true \
-Dlyng.profile.warmup=1 -Dlyng.profile.repeats=3 -Dlyng.profile.shuffle=true
```
- Dumps are saved to `lynglib/build/jfr_*.jfr` if the JVM supports Flight Recorder.
- The test also records GC counts/time and median time/heap deltas to `lynglib/build/book_alloc_profile.txt`.
## Toggling flags in tests
Flags are mutable at runtime, e.g.:
```kotlin
PerfFlags.ARG_BUILDER = false
val r1 = (Scope().eval(script) as ObjInt).value
PerfFlags.ARG_BUILDER = true
val r2 = (Scope().eval(script) as ObjInt).value
```
Reset flags at the end of a test to avoid impacting other tests.
## PIC diagnostics (optional)
Enable counters:
```kotlin
PerfFlags.PIC_DEBUG_COUNTERS = true
PerfStats.resetAll()
```
Available counters in `PerfStats`:
- Field PIC: `fieldPicHit`, `fieldPicMiss`, `fieldPicSetHit`, `fieldPicSetMiss`
- Method PIC: `methodPicHit`, `methodPicMiss`
- Index PIC: `indexPicHit`, `indexPicMiss`
- Locals: `localVarPicHit`, `localVarPicMiss`, `fastLocalHit`, `fastLocalMiss`
- Primitive ops: `primitiveFastOpsHit`
Print a summary at the end of a bench/test as needed. Remember to turn counters OFF after the test.
## A/B scenarios and guidance (JVM)
### Adaptive PIC (fields/methods)
- Flags: `FIELD_PIC=true`, `METHOD_PIC=true`, `FIELD_PIC_SIZE_4=false`, `METHOD_PIC_SIZE_4=false`, toggle `PIC_ADAPTIVE_2_TO_4` OFF vs ON.
- Benchmarks: `PicBenchmarkTest`, `MixedBenchmarkTest`, `PicAdaptiveABTest`.
- Expect wins at sites with >2 receiver shapes; counters should show fewer misses with adaptivity ON.
### Index PIC and size
- Flags: toggle `INDEX_PIC` OFF vs ON; then `INDEX_PIC_SIZE_4` OFF vs ON.
- Benchmarks: `ExpressionBenchmarkTest` (list indexing) and `IndexPicABTest` (string/map indexing).
- Expect wins when the same index shape recurs; counters should show higher `indexPicHit`.
### Index WRITE paths (Map and List)
- Flags: toggle `INDEX_PIC` OFF vs ON; then `INDEX_PIC_SIZE_4` OFF vs ON.
- Benchmark: `IndexWritePathABTest` (Map[String] put, List[Int] set) — writes results to `lynglib/build/index_write_ab_results.txt`.
- Direct fast‑paths are used on R‑value paths where safe and semantics‑preserving (e.g., optional‑chaining no‑ops on null receivers; bounds exceptions unchanged).
## Guidance per flag (JVM)
- Keep `RVAL_FASTPATH = true` unless debugging a suspected expression‑semantics issue.
- Use `SCOPE_POOL = true` only for benchmarks or once pooling passes the deep stress tests and broader validation; currently OFF by default.
- `FIELD_PIC` and `METHOD_PIC` should remain ON; they are validated with invalidation tests.
- Consider enabling `FIELD_PIC_SIZE_4`/`METHOD_PIC_SIZE_4` for sites with 3–4 receiver shapes; measure first.
- `PIC_ADAPTIVE_2_TO_4` is useful on polymorphic sites and may outperform fixed size 2 on mixed-shape workloads. Validate with `PicAdaptiveABTest`.
- `INDEX_PIC` is generally beneficial on JVM; leave ON when measuring index‑heavy workloads.
- `INDEX_PIC_SIZE_4` is ON by default on JVM as A/B showed consistent wins on String[Int] and Map[String] workloads. You can disable it by setting `PerfFlags.INDEX_PIC_SIZE_4 = false` if needed.
- `ARG_BUILDER` should remain ON; switch OFF only to get a baseline.
- `ARG_SMALL_ARITY_12` is experimental and OFF by default. Enable it only if your workload frequently calls functions with 9–12 arguments and A/B shows consistent wins.
### Workload‑specific recommendations (JVM)
- “Books”/documentation loads (BookTest): prefer simpler paths; in A/B these often benefit from the BOOKS preset (e.g., `ARG_BUILDER=false`, `SCOPE_POOL=false`, `INDEX_PIC=false`). Use `PerfProfiles.apply(PerfProfiles.Preset.BOOKS)` before the run and `restore(...)` after.
- Expression‑heavy benches: use the BENCH preset (PICs and R‑value fast‑paths enabled, `INDEX_PIC_SIZE_4=true`).
- Always verify with local A/B on your environment; rollback is a flag flip or applying BASELINE.
## Notes on correctness & safety
- Optional chaining semantics are preserved across fast paths.
- Visibility/mutability checks are enforced even on PIC fast‑paths.
- `frameId` is regenerated on each pooled frame borrow; stress tests verify no leakage under deep nesting/recursion.
## Cross‑platform
- Non‑JVM defaults keep `RVAL_FASTPATH=false` for now; other low‑risk flags may be ON.
- Once JVM path is fully validated and measured, add lightweight benches for JS/Wasm/Native and enable flags incrementally.
## Range fast iteration (experimental)
- Flag: `RANGE_FAST_ITER` (default OFF).
- When enabled and applicable, simple ascending integer ranges (`0..n`, `0..<n`) use a specialized non‑allocating iterator (`ObjFastIntRangeIterator`).
- Benchmark: `RangeIterationBenchmarkTest` records OFF/ON timings for inclusive, exclusive, reversed, negative, and empty ranges. Semantics are preserved; non‑int or complex ranges fall back to the generic iterator.
## Troubleshooting
- If a benchmark shows regressions, flip related flags OFF to isolate the source (e.g., `ARG_BUILDER`, `RVAL_FASTPATH`, `FIELD_PIC`, `METHOD_PIC`).
- Use `PIC_DEBUG_COUNTERS` to observe inline cache effectiveness.
- Ensure tests do not accidentally keep flags ON for subsequent tests; reset after each test.
## JVM micro-benchmark results (3× medians; OFF → ON)
Date: 2025-11-10 23:04 (local)
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|--------------------|----------------------------------------------|-----------------:|----------------:|:-------:|-------|
| ARG_BUILDER | CallMixedArityBenchmarkTest | 788.02 | 668.79 | 1.18× | Clear win on mixed arity |
| ARG_BUILDER | CallBenchmarkTest (simple calls) | 423.87 | 425.47 | 1.00× | Neutral on repeated simple calls |
| FIELD_PIC | PicBenchmarkTest::benchmarkFieldGetSetPic | 113.575 | 106.017 | 1.07× | Small but consistent win |
| METHOD_PIC | PicBenchmarkTest::benchmarkMethodPic | 251.068 | 149.439 | 1.68× | Large consistent win |
| RVAL_FASTPATH | ExpressionBenchmarkTest | 514.491 | 426.800 | 1.21× | Consistent win in expression chains |
| PRIMITIVE_FASTOPS | ArithmeticBenchmarkTest (int-sum) | 243.420 | 128.146 | 1.90× | Big win for integer addition |
| PRIMITIVE_FASTOPS | ArithmeticBenchmarkTest (int-cmp) | 210.385 | 168.534 | 1.25× | Moderate win for comparisons |
| SCOPE_POOL | CallPoolingBenchmarkTest | 505.778 | 366.737 | 1.38× | Single-threaded bench; per-thread ThreadLocal pool; default ON on JVM |
Notes:
- All results obtained from `[DEBUG_LOG] [BENCH]` outputs with three repeated Gradle test invocations per configuration; medians reported.
- JVM defaults (current): `ARG_BUILDER=true`, `PRIMITIVE_FASTOPS=true`, `RVAL_FASTPATH=true`, `FIELD_PIC=true`, `METHOD_PIC=true`, `SCOPE_POOL=true` (per‑thread ThreadLocal pool), `REGEX_CACHE=true`.
## Concurrency (multi‑core) pooling results (3× medians; OFF → ON)
Date: 2025-11-10 22:56 (local)
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|------------|--------------------------------------|-----------------:|----------------:|:-------:|-------|
| SCOPE_POOL | ConcurrencyCallBenchmarkTest (JVM) | 521.102 | 201.374 | 2.59× | Multithreaded workload on `Dispatchers.Default` with per‑thread ThreadLocal pool; workers=8, iters=15000/worker. |
Methodology:
- The test toggles `PerfFlags.SCOPE_POOL` within a single run and executes the same script across N worker coroutines scheduled on `Dispatchers.Default`.
- We executed the test three times via Gradle and computed medians from the printed `[DEBUG_LOG]` timings:
- OFF runs (ms): 532.442 | 521.102 | 474.386 → median 521.102
- ON runs (ms): 218.683 | 201.374 | 198.737 → median 201.374
- Speedup = OFF/ON.
Reproduce:
```
./gradlew :lynglib:jvmTest --tests "ConcurrencyCallBenchmarkTest" --rerun-tasks
```
## Next optimization steps (JVM)
Date: 2025-11-10 23:04 (local)
- PICs
- Widen METHOD_PIC to 3–4 entries with tiny LRU; keep invalidation on layout change; re-run `PicInvalidationJvmTest`.
- Micro fast-path for FIELD_PIC read-then-write pairs (`x = x + 1`) to reuse the resolved slot within one step.
- Locals and slots
- Pre-size `Scope` slot structures when compiler knows local/param counts; audit `EMIT_FAST_LOCAL_REFS` coverage.
- Re-run `LocalVarBenchmarkTest` to quantify gains.
- RVAL_FASTPATH coverage
- Cover primitive `ObjList` index reads, pure receivers in `FieldRef`, and assignment RHS where safe; add micro-benches to `ExpressionBenchmarkTest`.
- Collections and ranges
- Specialize `(Int..Int)` loops into tight counted loops (no intermediary objects).
- Add primitive-specialized `ObjList` ops (`map`, `filter`, `sum`, `contains`) under `PRIMITIVE_FASTOPS`.
- Regex and strings
- Cache compiled regex for string literals at compile time; add a tiny LRU for dynamic patterns behind `REGEX_CACHE`.
- Add `RegexBenchmarkTest` for repeated matches.
- JIT friendliness (Kotlin/JVM)
- Inline tiny helpers in hot paths, prefer arrays for internal buffers, finalize hot data structures where safe.
Validation matrix
- Always re-run: `CallBenchmarkTest`, `CallMixedArityBenchmarkTest`, `PicBenchmarkTest`, `ExpressionBenchmarkTest`, `ArithmeticBenchmarkTest`, `CallPoolingBenchmarkTest`, `DeepPoolingStressJvmTest`, `ConcurrencyCallBenchmarkTest` (3× medians when comparing).
- Keep full `:lynglib:jvmTest` green after each change.
## PIC update (4‑way METHOD_PIC) — JVM (3× medians; OFF → ON)
Date: 2025-11-11 00:16 (local)
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|-----------|-----------------------------------------------|-----------------:|----------------:|:-------:|-------|
| FIELD_PIC | PicBenchmarkTest::benchmarkFieldGetSetPic | 207.578 | 106.481 | 1.95× | Read→write loop; micro fast‑path groundwork present |
| METHOD_PIC| PicBenchmarkTest::benchmarkMethodPic | 273.478 | 182.226 | 1.50× | 4‑way PIC with move‑to‑front (was 2‑way before) |
Medians computed from three Gradle runs in this session; see `[DEBUG_LOG] [BENCH]` lines in test output.
## Locals/slots capacity (pre‑sizing hints) — JVM (3× medians; OFF → ON)
Date: 2025-11-11 13:19 (local)
| Optimization | Benchmark/Test | OFF config | ON config | OFF median (ms) | ON median (ms) | Speedup | Notes |
|-------------------------|-----------------------------|------------------------------------|------------------------------------|-----------------:|----------------:|:-------:|-------|
| Locals pre‑sizing + PIC | LocalVarBenchmarkTest | LOCAL_SLOT_PIC=OFF, FAST_LOCAL=OFF | LOCAL_SLOT_PIC=ON, FAST_LOCAL=ON | 472.129 | 370.871 | 1.27× | Compiler hint `params+4`; slot pre‑size; semantics unchanged |
Methodology:
- Each configuration executed three times via `:lynglib:jvmTest --tests "…" --rerun-tasks`; medians reported.
- Locals improvement stacks with per‑thread `SCOPE_POOL` and ARG fast paths.
## RVAL fast paths update — JVM (IndexRef and FieldRef) [3× medians; OFF → ON]
Date: 2025-11-11 13:19 (local)
New micro-benchmarks have been added to quantify the latest `RVAL_FASTPATH` extensions:
- Primitive `ObjList` index-read fast path in `IndexRef`.
- Conservative “pure receiver” evaluation in `FieldRef` (monomorphic, immutable receiver), preserving visibility/mutability checks and optional chaining semantics.
Benchmarks to run (each 3× OFF → ON):
- `ExpressionBenchmarkTest::benchmarkListIndexReads`
- `ExpressionBenchmarkTest::benchmarkFieldReadPureReceiver`
Reproduce (3× each; collect `[DEBUG_LOG] [BENCH]` lines and compute medians):
```
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
```
Once collected, add medians and speedups to the table below:
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|---------------|---------------------------------------------------|-----------------:|----------------:|:-------:|-------|
| RVAL_FASTPATH | ExpressionBenchmarkTest::benchmarkListIndexReads | 305.243 | 230.942 | 1.32× | Fast path in `IndexRef` for `ObjList` + `ObjInt` index |
| RVAL_FASTPATH | ExpressionBenchmarkTest::benchmarkFieldReadPureReceiver | 266.222 | 190.720 | 1.40× | Pure-receiver evaluation in `FieldRef` (monomorphic, immutable) |
Notes:
- Both benches toggle `PerfFlags.RVAL_FASTPATH` within a single run to produce OFF and ON timings under identical conditions.
- Correctness assertions ensure the loops are not optimized away.
- All semantics (visibility/mutability checks, optional chaining) remain intact; fast paths only skip interim `ObjRecord` traffic when safe.
## ARG_BUILDER — splat fast‑path (3× medians; OFF → ON)
Date: 2025-11-11 13:12 (local)
Environment: Gradle 8.7; JVM (JDK as configured by toolchain); single‑threaded test execution; stdout enabled.
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|-------------|-----------------------------------|-----------------:|----------------:|:-------:|-------|
| ARG_BUILDER | CallSplatBenchmarkTest (splat) | 613.689 | 463.593 | 1.32× | Single‑splat fast‑path returns underlying list directly; avoids intermediate copies |
Inputs (3×):
- OFF runs (ms): 613.689 | 629.604 | 612.361 → median 613.689
- ON runs (ms): 453.752 | 463.593 | 468.844 → median 463.593
Reproduce (3×):
```
./gradlew :lynglib:jvmTest --tests "CallSplatBenchmarkTest" --rerun-tasks
```
## Phase A consolidation (JVM) — 3× medians updated
Date: 2025-11-11 13:48 (local)
Environment:
- JDK: OpenJDK 20.0.2.1 (Amazon Corretto 20.0.2.1+10-FR)
- Gradle: 8.7
- OS/Arch: macOS 14.8.1 (aarch64)
### ARG_BUILDER
| Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|----------------------------------|-----------------:|----------------:|:-------:|-------|
| CallMixedArityBenchmarkTest | 866.681 | 717.439 | 1.21× | Small-arity 0–8 fast path + builder; correctness preserved |
| CallSplatBenchmarkTest (splat) | 600.880 | 459.706 | 1.31× | Single-splat fast path returns underlying list; avoids copies |
Inputs (3×):
- Mixed arity OFF: 874.088291 | 866.680959 | 858.577125 → median 866.680959
- Mixed arity ON: 731.308625 | 706.440125 | 717.438542 → median 717.438542
- Splat OFF: 600.268625 | 607.849416 | 600.879666 → median 600.879666
- Splat ON: 459.706375 | 449.950166 | 461.815167 → median 459.706375
### RVAL_FASTPATH (new coverage)
| Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|--------------------------------------------------|-----------------:|----------------:|:-------:|-------|
| ExpressionBenchmarkTest::benchmarkListIndexReads | 299.366 | 218.812 | 1.37× | IndexRef fast path for ObjList + ObjInt |
| ExpressionBenchmarkTest::benchmarkFieldReadPureReceiver | 268.315 | 186.032 | 1.44× | Pure-receiver evaluation in FieldRef (monomorphic, immutable) |
Inputs (3×):
- ListIndex OFF: 291.344 | 310.717167 | 299.365709 → median 299.365709
- ListIndex ON: 217.795375 | 221.504166 | 218.812042 → median 218.812042
- FieldRead OFF: 267.2775 | 274.355208 | 268.315125 → median 268.315125
- FieldRead ON: 189.599333 | 186.031791 | 182.069167 → median 186.031791
### Locals/slots capacity (precise hints)
| Benchmark/Test | OFF config | ON config | OFF median (ms) | ON median (ms) | Speedup | Notes |
|---------------------------|------------------------------------|------------------------------------|-----------------:|----------------:|:-------:|-------|
| LocalVarBenchmarkTest | LOCAL_SLOT_PIC=OFF, FAST_LOCAL=OFF | LOCAL_SLOT_PIC=ON, FAST_LOCAL=ON | 446.018 | 347.964 | 1.28× | Precise capacity hints + fast-locals coverage |
Inputs (3×):
- Locals OFF: 470.575041 | 441.89625 | 446.017833 → median 446.017833
- Locals ON: 370.664208 | 345.615541 | 347.964291 → median 347.964291
Methodology:
- Each test executed three times via Gradle with stdout enabled; medians computed from `[DEBUG_LOG] [BENCH]` lines.
- Full JVM tests and stress benches remain green in this cycle.
## Phase B — List ops specialization (PRIMITIVE_FASTOPS) — 3× medians (OFF → ON)
Date: 2025-11-11 13:48 (local)
Environment:
- JDK: OpenJDK 20.0.2.1 (Amazon Corretto 20.0.2.1+10-FR)
- Gradle: 8.7
- OS/Arch: macOS 14.8.1 (aarch64)
| Optimization | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|---------------------|------------------------------------------|-----------------:|----------------:|:-------:|-------|
| PRIMITIVE_FASTOPS | ListOpsBenchmarkTest::benchmarkSumInts | 324.805 | 144.908 | 2.24× | ObjList.sum fast path for int lists; generic fallback preserved |
| PRIMITIVE_FASTOPS | ListOpsBenchmarkTest::benchmarkContainsInts | 440.414 | 415.476 | 1.06× | ObjList.contains fast path when searching ObjInt in int list |
Inputs (3×):
- list-sum OFF: 332.863417 | 323.491625 | 324.804083 → median 324.804083
- list-sum ON: 144.907833 | 148.870792 | 126.418542 → median 144.907833
- list-contains OFF: 440.413709 | 440.368333 | 441.4365 → median 440.413709
- list-contains ON: 416.465292 | 412.283291 | 415.475833 → median 415.475833
Methodology:
- Each test executed three times via Gradle; medians computed from `[DEBUG_LOG] [BENCH]` lines.
- Changes are fully guarded by `PerfFlags.PRIMITIVE_FASTOPS`; semantics preserved (null on empty sum; generic fallback on mixed types).
### Phase B — Ranges for-in lowering (PRIMITIVE_FASTOPS) — 3× medians (OFF → ON)
Date: 2025-11-11 13:48 (local)
Environment:
- JDK: OpenJDK 20.0.2.1 (Amazon Corretto 20.0.2.1+10-FR)
- Gradle: 8.7
- OS/Arch: macOS 14.8.1 (aarch64)
| Optimization | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|---------------------|------------------------------------------|-----------------:|----------------:|:-------:|-------|
| PRIMITIVE_FASTOPS | RangeBenchmarkTest::benchmarkIntRangeForIn | 1705.299 | 788.974 | 2.16× | Tight counted loop for (Int..Int) for-in; preserves semantics |
Inputs (3×):
- range-for-in OFF: 1705.298958 | 1684.357708 | 1735.880917 → median 1705.298958
- range-for-in ON: 794.178458 | 778.741834 | 788.973625 → median 788.973625
Methodology:
- Each configuration executed three times via Gradle; medians computed from `[DEBUG_LOG] [BENCH]` lines.
- Lowering is guarded by `PerfFlags.PRIMITIVE_FASTOPS` and applies only when the source is an `ObjRange` with int bounds; otherwise falls back to generic iteration.
## Phase B — Regex caching (REGEX_CACHE) — 3× medians (OFF → ON)
Date: 2025-11-11 13:48 (local)
Environment:
- JDK: OpenJDK 20.0.2.1 (Amazon Corretto 20.0.2.1+10-FR)
- Gradle: 8.7
- OS/Arch: macOS 14.8.1 (aarch64)
| Flag | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|--------------|---------------------------------------------------|-----------------:|----------------:|:-------:|-------|
| REGEX_CACHE | RegexBenchmarkTest::benchmarkLiteralPatternMatches | 378.246 | 275.890 | 1.37× | Caches compiled regex for identical literal pattern per iteration |
| REGEX_CACHE | RegexBenchmarkTest::benchmarkDynamicPatternMatches | 514.944 | 229.006 | 2.25× | Two dynamic patterns alternate; cache size sufficient to retain both |
Inputs (1× here; can extend to 3× on request):
- regex-literal OFF: 378.245916; ON: 275.889541
- regex-dynamic OFF: 514.944167; ON: 229.005834
Methodology:
- Each benchmark toggles `PerfFlags.REGEX_CACHE` inside a single test and prints `[DEBUG_LOG]` timings for OFF and ON runs under identical conditions. We recorded one set of OFF/ON timings here; we can extend to 3× medians if required for publication.
- The cache is a tiny size-bounded map (64 entries) activated only when `PerfFlags.REGEX_CACHE` is true. Defaults remain OFF.
## JIT tweaks (Round 1) — quick gains snapshot (locals, ranges, list ops)
Date: 2025-11-11 21:05 (local)
Scope: fast confirmation of overall gain using current configuration; focused on locals, ranges, and list ops. Each test prints OFF → ON timings in a single run. We executed the benches via Gradle with stdout enabled and single test fork.
Environment:
- Gradle: 8.7 (stdout enabled, maxParallelForks=1)
- JVM: as configured by toolchain for this project
- OS/Arch: per developer machine (unchanged from prior sections)
Reproduce:
```
./gradlew :lynglib:jvmTest --tests LocalVarBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests RangeBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ListOpsBenchmarkTest --rerun-tasks
```
Results (representative runs; OFF → ON):
- Local variables — LOCAL_SLOT_PIC + EMIT_FAST_LOCAL_REFS
- Run 1: 468.407 ms → 367.277 ms (≈ 1.28×)
- Run 2: 447.031 ms → 346.126 ms (≈ 1.29×)
- Ranges for‑in — PRIMITIVE_FASTOPS (tight counted loop for (Int..Int))
- 1731.780 ms → 799.023 ms (≈ 2.17×)
- List ops — PRIMITIVE_FASTOPS
- sum(int list): 318.943 ms → 148.571 ms (≈ 2.15×)
- contains(int in int list): 440.013 ms → 412.450 ms (≈ 1.07×)
Summary: All three areas improved with optimizations ON; no regressions observed in these runs. For publication‑grade stability, run each test 3× and report medians (see sections below for methodology and previous median tables).
## Additional tweaks — verification snapshot (Index write fast‑path, List literal pre‑size, Regex LRU)
Date: 2025-11-11 21:31 (local)
Scope: Implemented three semantics‑neutral optimizations and verified they are green across targeted and broader JVM benches.
What changed (guarded by flags where applicable):
- RVAL_FASTPATH: Index write fast‑path
- `IndexRef.setAt`: direct path for `ObjList` + `ObjInt` (`list[i] = value`) mirrors the read fast‑path. Optional chaining semantics preserved; bounds exceptions propagate unchanged.
- RVAL_FASTPATH: List literal pre‑sizing
- `ListLiteralRef.get`: pre‑counts element entries and uses `ArrayList` with capacity hint; for spreads of `ObjList`, uses `ensureCapacity` before bulk add. Evaluation order unchanged.
- REGEX_CACHE: LRU‑like behavior
- `RegexCache`: emulates access‑order LRU within a tiny bounded map (`MAX=64`) by moving accessed entries to the tail; improves alternating‑pattern scenarios. Only active when `PerfFlags.REGEX_CACHE` is true.
Reproduce quick verification (1× runs):
```
./gradlew :lynglib:jvmTest --tests ExpressionBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ListOpsBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests RegexBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests PicBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests PicInvalidationJvmTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests LocalVarBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ConcurrencyCallBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests DeepPoolingStressJvmTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests MultiThreadPoolingStressJvmTest --rerun-tasks
```
Observation: All listed tests green in this cycle; no behavioral regressions observed. For the new paths (index write, list literal), performance was neutral‑to‑positive in smoke runs; Regex benches remained positive or neutral with the LRU behavior. For publication‑grade medians, extend to 3× per test as in earlier sections.
## Sanity matrix (JVM) — quick OFF→ON runs
Date: 2025-11-11 21:59 (local)
Scope: Final Round 1 sanity sweep across JVM micro‑benches and stress tests to confirm that optimizations ON do not regress performance vs OFF in representative scenarios. Each benchmark prints `[DEBUG_LOG] [BENCH]` timings for OFF → ON within a single run. This section records a quick pass confirmation (not 3× medians) and reproduction commands.
Environment:
- Gradle: 8.7 (stdout enabled, maxParallelForks=1)
- JVM: as configured by the project toolchain
- OS/Arch: macOS 14.x (aarch64)
Benches covered (all green; no regressions observed in these runs):
- Calls/Args: `CallBenchmarkTest`, `CallMixedArityBenchmarkTest` (ARG_BUILDER)
- PICs: `PicBenchmarkTest` (field/method); `PicInvalidationJvmTest` correctness reconfirmed
- Expressions/Arithmetic: `ExpressionBenchmarkTest`, `ArithmeticBenchmarkTest` (RVAL_FASTPATH, PRIMITIVE_FASTOPS)
- Ranges: `RangeBenchmarkTest` (PRIMITIVE_FASTOPS counted loop)
- List ops: `ListOpsBenchmarkTest` (PRIMITIVE_FASTOPS specializations)
- Regex: `RegexBenchmarkTest` (REGEX_CACHE with LRU behavior)
- Locals: `LocalVarBenchmarkTest` (LOCAL_SLOT_PIC + FAST_LOCAL)
- Concurrency/Pooling: `ConcurrencyCallBenchmarkTest`, `DeepPoolingStressJvmTest`, `MultiThreadPoolingStressJvmTest` (SCOPE_POOL per‑thread)
Reproduce (examples):
```
./gradlew :lynglib:jvmTest --tests CallBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests CallMixedArityBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests PicBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests PicInvalidationJvmTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ExpressionBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ArithmeticBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests RangeBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ListOpsBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests RegexBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests LocalVarBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests ConcurrencyCallBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests DeepPoolingStressJvmTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests MultiThreadPoolingStressJvmTest --rerun-tasks
```
Summary:
- All listed tests passed in this sanity sweep.
- For each benchmark’s OFF → ON printouts examined during this pass, ON was equal or faster than OFF; no ON<OFF regressions were observed.
- For publication‑grade numbers, use the 3× medians methodology outlined earlier in this document. The existing median tables in previous sections remain representative, and the additional tweaks (Index write, List literal pre‑size, Regex LRU, Field PIC 4‑way + read→write reuse, mixed Int/Real fast‑ops) remained neutral‑to‑positive.
## Quick snapshot — IndexRef PIC + negative miss cache (JVM) — 3× medians (OFF → ON)
Date: 2025-11-11 22:32 (local)
Scope
- Confirm that the latest changes — IndexRef read/write PIC (stacked on RVAL_FASTPATH) and safe catch‑and‑cache negative entries for Field/Method PICs — do not regress performance. We collected 3× medians for the two expression sub‑benches that are most sensitive to RVAL paths and cross‑checked PICs and ranges.
Environment
- Gradle: 8.7 (stdout enabled, maxParallelForks=1)
- JVM: project toolchain default
- OS/Arch: macOS 14.x (aarch64)
Results (3× medians)
| Area | Benchmark/Test | OFF median (ms) | ON median (ms) | Speedup | Notes |
|------|-----------------|-----------------:|----------------:|:-------:|-------|
| RVAL_FASTPATH | ExpressionBenchmarkTest::benchmarkListIndexReads | 304.282 | 229.168 | 1.33× | IndexRef direct fast‑path for ObjList+ObjInt; 4‑way Index PIC handles polymorphic cases |
| RVAL_FASTPATH | ExpressionBenchmarkTest::benchmarkFieldReadPureReceiver | 275.122 | 194.876 | 1.41× | Monomorphic, immutable receiver path; preserves visibility/optional semantics |
Cross‑checks (from the same session, 1× quick)
- PicBenchmarkTest::benchmarkFieldGetSetPic — OFF 203.701 ms → ON 117.129 ms (≈1.74×)
- PicBenchmarkTest::benchmarkMethodPic — OFF 280.806 ms → ON 202.613 ms (≈1.39×)
- RangeBenchmarkTest::benchmarkIntRangeForIn — OFF 1762.425 ms → ON 806.898 ms (≈2.18×)
Reproduce
```
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkListIndexReads" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
./gradlew :lynglib:jvmTest --tests "ExpressionBenchmarkTest.benchmarkFieldReadPureReceiver" --rerun-tasks
./gradlew :lynglib:jvmTest --tests PicBenchmarkTest --rerun-tasks
./gradlew :lynglib:jvmTest --tests RangeBenchmarkTest --rerun-tasks
```
Notes
- Negative caches are installed only after a real miss throws (cache‑after‑miss), preserving error semantics and invalidation on `layoutVersion` changes.
- IndexRef PIC augments the existing direct path and uses move‑to‑front promotion; it is keyed on `(classId, layoutVersion)` like other PICs.
+58
View File
@@ -0,0 +1,58 @@
# JVM-only Performance Optimization Plan (Saved)
[//]: # (excludeFromIndex)
Date: 2025-11-10 22:14 (local)
This document captures the agreed next optimization steps so we can restore the plan later if needed.
## Objectives
- Reduce overhead on the call/argument path.
- Extend and harden PIC performance (fields/methods/locals).
- Improve R-value fast paths and interpreter hot nodes (loops, ranges, lists).
- Make scope frame pooling thread-safe on JVM so it can be enabled by default later.
- Keep semantics correct and all JVM tests green.
## Prioritized tasks (now)
1) Call/argument path: fewer allocs, tighter fast paths
- Extend small-arity zero-alloc path to 6–8 args; benchmark with `CallMixedArityBenchmarkTest`.
- Splat handling: fast-path single-list splats; benchmark with `CallSplatBenchmarkTest`.
- Arg builder micro-optimizations: capacity hints, avoid redundant copies, inline simple branches.
- Optional-chaining fast return (`SKIP_ARGS_ON_NULL_RECEIVER`) coverage audit, add A/B bench.
2) Scope frame pooling: per-thread safety on JVM
- Replace global deque with ThreadLocal pool on JVM (and Android) actuals.
- Keep `frameId` uniqueness and pool size cap.
- Verify with `DeepPoolingStressJvmTest`, `CallPoolingBenchmarkTest`, and spot benches.
- Do NOT flip default yet; keep `SCOPE_POOL=false` unless explicitly approved.
## Next tasks (queued)
3) PICs: cheaper misses, broader hits
- Method PIC 2→3/4 entries (tiny LRU); validate with `PicInvalidationJvmTest`.
- Field PIC micro-fast path for read-then-write pairs.
4) Locals and slots
- Ensure `EMIT_FAST_LOCAL_REFS` coverage across compiler sites.
- Pre-size `slots`/`nameToSlot` when local counts are known; re-run `LocalVarBenchmarkTest`.
5) R-value fast path coverage
- Cover index reads on primitive lists, pure receivers, assignment RHS where safe.
- Add benches in `ExpressionBenchmarkTest`.
6) Collections & ranges
- Tight counted loop for `(Int..Int)` in `for`.
- Primitive-specialized `ObjList` ops (`map`, `filter`, `sum`, `contains`) under `PRIMITIVE_FASTOPS`.
7) Regex and string ops
- Cache compiled regex for string literals at compile time; tiny LRU for dynamic patterns under a new `REGEX_CACHE` flag.
8) JIT micro-tweaks
- Inline tiny helpers; prefer arrays for hot buffers; finalize hot classes where safe.
## Validation matrix
- Always re-run: `CallBenchmarkTest`, `CallMixedArityBenchmarkTest`, `PicBenchmarkTest`, `ExpressionBenchmarkTest`, `ArithmeticBenchmarkTest`, `CallPoolingBenchmarkTest`, `DeepPoolingStressJvmTest`.
- Use 3× medians where comparing flags; keep `:lynglib:jvmTest` green.
## Notes
- All risky changes remain flag-guarded and JVM-only where applicable.
- Documentation and perf tables updated after each cycle.
+95
View File
@@ -0,0 +1,95 @@
# Map literals — refined proposal
[//]: # (excludeFromIndex)
Implement JavaScript-like literals for maps. The syntax and semantics align with named arguments in function calls, but map literals are expressions that construct `Map` values.
Keys can be either:
- string literals: `{ "some key": value }`, or
- identifiers: `{ name: expr }`, where the key becomes the string `"name"`.
Identifier shorthand inside map literals is supported:
- `{ name: }` desugars to `{ "name": name }`.
Property access sugar is not provided for maps: use bracket access only, e.g. `m["a"]`, not `m.a`.
Examples:
```lyng
val x = 2
val m = { "a": 1, x: x*10, y: }
assertEquals(1, m["a"]) // string-literal key
assertEquals(20, m["x"]) // identifier key
assertEquals(2, m["y"]) // identifier shorthand
```
Spreads (splats) in map literals are allowed and merged left-to-right with “rightmost wins” semantics:
```lyng
val base = { a: 1, b: 2 }
val m = { a: 0, ...base, b: 3, c: 4 }
assertEquals(1, m["a"]) // base overwrites a:0
assertEquals(3, m["b"]) // literal overwrites spread
assertEquals(4, m["c"]) // new key
```
Trailing commas are allowed (optional):
```lyng
val m = {
"a": 1,
b: 2,
...other,
}
```
Duplicate keys among literal entries (including identifier shorthand) are a compile-time error:
```lyng
{ foo: 1, "foo": 2 } // error: duplicate key "foo"
{ foo:, foo: 2 } // error: duplicate key "foo"
```
Spreads are evaluated at runtime. Overlaps from spreads are resolved by last write wins. If a spread is not a map, or yields a map with non-string keys, it’s a runtime error.
Merging with `+`/`+=` and entries:
```lyng
("1" => 10) + ("2" => 20) // Map("1"=>10, "2"=>20)
{ "1": 10 } + ("2" => 20) // same
{ "1": 10 } + { "2": 20 } // same
var m = { "a": 1 }
m += ("b" => 2) // m = { "a":1, "b":2 }
```
Rightmost wins on duplicates consistently across spreads and merges. All map merging operations require string keys; encountering a non-string key during merge is a runtime error.
Empty map literal `{}` is not supported to avoid ambiguity with blocks/lambdas. Use `Map()` for an empty map.
Lambda disambiguation
- A `{ ... }` with typed lambda parameters must have a top-level `->` in its header. The compiler disambiguates by looking for a top-level `->`. If none is found, it attempts to parse a map literal; if that fails, it is parsed as a lambda or block.
Grammar (EBNF)
```
ws = zero or more whitespace (incl. newline/comments)
map_literal = '{' ws map_entries ws '}'
map_entries = map_entry ( ws ',' ws map_entry )* ( ws ',' )?
map_entry = map_key ws ':' ws map_value_opt
| '...' ws expression
map_key = string_literal | ID
map_value_opt = expression | ε // ε allowed only if map_key is ID
```
Notes:
- Identifier shorthand (`id:`) is allowed only for identifiers, not string-literal keys.
- Spreads accept any expression; at runtime it must yield a `Map` with string keys.
- Duplicate keys are detected at compile time among literal keys; spreads are merged at runtime with last-wins.
Rationale
- The `{ name: value }` style is familiar and ergonomic.
- Disambiguation with lambdas leverages the required `->` in typed lambda headers.
- Avoiding `m.a` sidesteps method/field shadowing and keeps semantics clear.
+70
View File
@@ -0,0 +1,70 @@
# Named arguments in calls
[//]: # (excludeFromIndex)
Extend function/method calls to allow setting arguments by name using colon syntax at call sites. This is especially convenient with many parameters and default values.
Examples:
```lyng
fun test(a="foo", b="bar", c="bazz") { [a, b, c] }
assertEquals(test(b: "b"), ["foo", "b", "bazz"])
assertEquals(test("a", c: "c"), ["a", "bar", "c"])
```
Rules:
- Named arguments are optional. If named arguments are present, their order is not important.
- Named arguments must follow positional arguments; positional arguments cannot follow named ones (the only exception is the syntactic trailing block outside parentheses, see below).
- A named argument cannot reassign a parameter already set positionally.
- If the last parameter is already assigned by a named argument (or named splat), the trailing-lambda rule must NOT apply: a following `{ ... }` after the call is an error.
Rationale for using `:` instead of `=` in calls: in Lyng, assignment `=` is an expression; using `:` avoids ambiguity and keeps declarations (`name: Type`) distinct from call sites, where casting uses `as` / `as?`.
Migration note: earlier drafts/examples used `name = value`. The final syntax is `name: value` at call sites.
## Extended call argument splats: named splats
With named arguments, splats (`...`) are extended to support maps as named splats. When a splat evaluates to a Map, its entries provide name→value assignments:
```lyng
fun test(a="a", b="b", c="c", d="d") { [a, b, c, d] }
assertEquals(test("A?", ...Map("d" => "D!", "b" => "B!")), ["A?", "B!", "c", "D!"])
```
Constraints for named splats:
- Only string keys are allowed in map splats; otherwise, a clean error is thrown.
- Named splats cannot reassign parameters already set (positionally or by earlier named arguments/splats).
- Named splats follow the same ordering as named arguments: they must appear after all positional arguments and positional splats.
## Trailing-lambda interaction
Lyng supports a syntactic trailing block after a call: `f(args) { ... }`. With named args/splats, if the last parameter is already assigned by name, the trailing block must not apply and the call is an error:
```lyng
fun f(x, onDone) { onDone(x) }
f(x: 1) { 42 } // ERROR: last parameter already assigned by name
f(1) { 42 } // OK
```
## Errors (non-exhaustive)
- Positional argument after any named argument inside parentheses: error.
- Positional splat after any named argument: error.
- Duplicate named assignment (directly or via map splats): error.
- Unknown parameter name in a named argument/splat: error.
- Map splat with non-string keys: error.
- Attempt to target the ellipsis parameter by name: error.
## Notes
- Declarations continue to use `:` for types, while call sites use `:` for named arguments and `as` / `as?` for type casts/checks.
+25
View File
@@ -0,0 +1,25 @@
#!/bin/env lyng
import lyng.io.fs
import lyng.stdlib
val files = Path("../..").list().toList()
// most long is longest?
val longestNameLength = files.maxOf { it.name.length }
// testdoc
fun test() {
22
}
val format = "%"+(longestNameLength+1) +"s %d"
for( f in files )
{
var name = f.name
if( f.isDirectory() )
name += "/"
println( format(name, f.size()) )
}
@@ -16,9 +16,16 @@ fun naiveCountHappyNumbers() {
count
}
import lyng.time
//
// After all optimizations it takes ~120ms.
// After all optimizations it took ~120ms - before.
//
val found = naiveCountHappyNumbers()
println("Found happy numbers: "+found)
assert( found == 55252 )
for( r in 1..900 ) {
val start = Instant.now()
val found = naiveCountHappyNumbers()
println("Found happy numbers: %d time %s"(found, Instant.now() - start))
assert( found == 55252 )
delay(0.05)
}
+10
View File
@@ -0,0 +1,10 @@
#!/bin/env lyng
val переменная = "значение"
val data = { greeting: "привет", переменная:, поле: "содержимое" }
assertEquals(data["переменная"], "значение")
assertEquals(data["поле"], "содержимое")
println(data)
Regular → Executable
+1 -1
View File
@@ -1,7 +1,7 @@
#!/bin/env lyng
/*
Calculate the limit of Sum( f(n) )
until it reaches asymptotic limit 0.00001% change
return null or found limit
*/
fun findSumLimit(f) {
+3 -2
View File
@@ -48,6 +48,7 @@ $$ \ln(1+x)=x-{\dfrac {x^{2}}{2}}+{\dfrac {x^{3}}{3}}-\cdots =\sum \limits _{n=0
}
assert( почти_равны( 0.0005, 0.000501 ) )
Во многих случаях вычисление $n+1$ члена значительно проще cчитается от предыдущего члена, в нашем случае это можно было бы записать через итератор, что мы вскоре добавим.
Во многих случаях вычисление $n+1$ члена значительно проще cчитается от предыдущего члена, в нашем случае это можно было бы записать через итератор, см [Iterable] и `flow` в [parallelism].
(продолжение следует)
[Iterable]: ../Iterable.md
[parallelism]: ../parallelism.md
+75
View File
@@ -0,0 +1,75 @@
# Scopes and Closures: resolution and safety
This page documents how name resolution works with `ClosureScope`, how to avoid recursion pitfalls, and how to safely capture and execute callbacks that need access to outer locals.
## Why this matters
Name lookup across nested scopes and closures can accidentally form recursive resolution paths or hide expected symbols (outer locals, module/global functions). The rules below ensure predictable resolution and prevent infinite recursion.
## Resolution order in ClosureScope
When evaluating an identifier `name` inside a closure, `ClosureScope.get(name)` resolves in this order:
1. Closure frame locals and arguments
2. Captured receiver (`closureScope.thisObj`) instance/class members
3. Closure ancestry locals + each frame’s `thisObj` members (cycle‑safe)
4. Caller `this` members
5. Caller ancestry locals + each frame’s `thisObj` members (cycle‑safe)
6. Module pseudo‑symbols (e.g., `__PACKAGE__`) from the nearest `ModuleScope`
7. Direct module/global fallback (nearest `ModuleScope` and its parent/root scope)
8. Final fallback: base local/parent lookup for the current frame
This preserves intuitive visibility (locals → captured receiver → closure chain → caller members → caller chain → module/root) while preventing infinite recursion between scope types.
## Use raw‑chain helpers for ancestry walks
When authoring new scope types or advanced lookups, avoid calling virtual `get` while walking parents. Instead, use the non‑dispatch helpers on `Scope`:
- `chainLookupIgnoreClosure(name)`
- Walk raw `parent` chain and check only per‑frame locals/bindings/slots.
- Ignores overridden `get` (e.g., in `ClosureScope`). Cycle‑safe.
- `chainLookupWithMembers(name)`
- Like above, but after locals/bindings it also checks each frame’s `thisObj` members.
- Ignores overridden `get`. Cycle‑safe.
- `baseGetIgnoreClosure(name)`
- For the current frame only: check locals/bindings, then walk raw parents (locals/bindings), then fallback to this frame’s `thisObj` members.
These helpers avoid ping‑pong recursion and make structural cycles harmless (lookups terminate).
## Preventing structural cycles
- Don’t construct parent chains that can point back to a descendant.
- A debug‑time guard throws if assigning a parent would create a cycle; keep it enabled for development builds.
- Even with a cycle, chain helpers break out via a small `visited` set keyed by `frameId`.
## Capturing lexical environments for callbacks
For dynamic objects or custom builders, capture the creator’s lexical scope so callbacks can see outer locals/parameters:
1. Use `snapshotForClosure()` on the caller scope to capture locals/bindings/slots and parent.
2. Store this snapshot and run callbacks under `ClosureScope(callScope, captured)`.
Kotlin sketch:
```kotlin
val captured = scope.snapshotForClosure()
val execScope = ClosureScope(currentCallScope, captured)
callback.execute(execScope)
```
This ensures expressions like `contractName` used inside dynamic `get { name -> ... }` resolve to outer variables defined at the creation site.
## Closures in coroutines (launch/flow)
- The closure frame still prioritizes its own locals/args.
- Outer locals declared before suspension points remain visible through slot‑aware ancestry lookups.
- Global functions like `delay(ms)` and `yield()` are resolved via module/root fallbacks from within closures.
Tip: If a closure unexpectedly cannot see an outer local, check whether an intermediate runtime helper introduced an extra call frame; the built‑in lookup already traverses caller ancestry, so prefer the standard helpers rather than custom dispatch.
## Local variable references and missing symbols
- Unqualified identifier resolution first prefers locals/bindings/slots before falling back to `this` members.
- If neither locals nor members contain the symbol, missing field lookups map to `SymbolNotFound` (compatibility alias for `SymbolNotDefinedException`).
## Performance notes
- The `visited` sets used for cycle detection are tiny and short‑lived; in typical scripts the overhead is negligible.
- If profiling shows hotspots, consider limiting ancestry depth in your custom helpers or using small fixed arrays instead of hash sets—only for extremely hot code paths.
## Dos and Don’ts
- Do use `chainLookupIgnoreClosure` / `chainLookupWithMembers` for ancestry traversals.
- Do maintain the resolution order above for predictable behavior.
- Don’t call virtual `get` while walking parents; it risks recursion across scope types.
- Don’t attach instance scopes to transient/pool frames; bind to a stable parent scope instead.
+46
View File
@@ -0,0 +1,46 @@
# Lyng serialization
Lyng has builting binary bit-effective serialization format, called Lynon for LYng Object Notation. It is typed, binary, implements caching, automatic compression, variable-length ints, one-bit Booleans an many nice features.
It is as simple as:
import lyng.serialization
val text = "
We hold these truths to be self-evident, that all men are created equal,
that they are endowed by their Creator with certain unalienable Rights,
that among these are Life, Liberty and the pursuit of Happiness.
"
val encodedBits = Lynon.encode(text)
// decode bits source:
assertEquals( text, Lynon.decode(encodedBits) )
// compression was used automatically
assert( text.length > encodedBits.toBuffer().size )
>>> void
Any class you create is serializable by default; lynon serializes first constructor fields, then any `var` member fields:
import lyng.serialization
class Point(x,y)
val p = Lynon.decode( Lynon.encode( Point(5,6) ) )
assertEquals( 5, p.x )
assertEquals( 6, p.y )
>>> void
just as expected.
Important is to understand that normally `Lynon.decode` wants [BitBuffer], as `Lynon.encode` produces. If you have the regular [Buffer], be sure to convert it:
buffer.toBitInput()
this possibly creates extra zero bits at the end, as bit content could be shorter than byte-grained but for the Lynon format it does not make sense. Note that when you serialize [BitBuffer], exact number of bits is written. To convert bit buffer to bytes:
Lynon.encode("hello").toBuffer()
(topic is incomplete and under construction)
+77
View File
@@ -0,0 +1,77 @@
# TextMate bundle
[//]: # (excludeFromIndex)
The TextMate-format bundle contains a syntax definition for initial language support in
popular editors that understand TextMate grammars: TextMate, Visual Studio Code, Sublime Text, etc.
- [Download TextMate Bundle for Lyng](https://lynglang.com/distributables/lyng-textmate.zip)
> Note for IntelliJ-based IDEs (IntelliJ IDEA, Fleet, etc.): although you can import TextMate
> bundles there (Settings/Preferences → Editor → TextMate Bundles), we strongly recommend using the
> dedicated plugin instead — it provides much better support (formatting, smart enter, background
> analysis, etc.). See: [IDEA Plugin](#/docs/idea_plugin.md).
## Visual Studio Code
VS Code uses TextMate grammars packaged as extensions. A minimal local extension is easy to set up:
1) Download and unzip the bundle above. Inside you will find the grammar file (usually
`*.tmLanguage.json` or `*.tmLanguage` plist).
2) Create a new folder somewhere, e.g. `lyng-textmate-vscode/` with the following structure:
```
lyng-textmate-vscode/
package.json
syntaxes/
lyng.tmLanguage.json # copy the grammar file here (rename if needed)
```
3) Put this minimal `package.json` into that folder (adjust file names if needed):
```
{
"name": "lyng-textmate",
"displayName": "Lyng (TextMate grammar)",
"publisher": "local",
"version": "0.0.1",
"engines": { "vscode": "^1.70.0" },
"contributes": {
"languages": [
{ "id": "lyng", "aliases": ["Lyng"], "extensions": [".lyng"] }
],
"grammars": [
{
"language": "lyng",
"scopeName": "source.lyng",
"path": "./syntaxes/lyng.tmLanguage.json"
}
]
}
}
```
4) Open a terminal in `lyng-textmate-vscode/` and run:
```
code --install-extension .
```
Alternatively, open the folder in VS Code and press F5 to run an Extension Development Host.
5) Reload VS Code. Files with the `.lyng` extension should now get Lyng highlighting.
## Sublime Text 3/4
1) Download and unzip the bundle.
2) In Sublime Text, use “Preferences → Browse Packages…”, then copy the unzipped bundle
to a folder like `Packages/Lyng/`.
3) Open a `.lyng` file; Sublime should pick up the syntax automatically. If not, use
“View → Syntax → Lyng”.
## TextMate 2
1) Download and unzip the bundle.
2) Double‑click the `.tmBundle`/grammar package or drag it onto TextMate to install, or place
it into `~/Library/Application Support/TextMate/Bundles/`.
3) Restart TextMate if needed and open a `.lyng` file.
+366 -89
View File
@@ -1,8 +1,3 @@
---
gitea: none
include_toc: true
---
# Lyng tutorial
Lyng is a very simple language, where we take only most important and popular features from
@@ -11,12 +6,15 @@ In other word, the code usually works as expected when you see it. So, nothing u
__Other documents to read__ maybe after this one:
- [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md)
- [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md), [Scopes and Closures](scopes_and_closures.md)
- [OOP notes](OOP.md), [exception handling](exceptions_handling.md)
- [math in Lyng](math.md)
- [math in Lyng](math.md), [the `when` statement](when.md)
- [time](time.md) and [parallelism](parallelism.md)
- [parallelism] - multithreaded code, coroutines, etc.
- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md)
- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
- Some class
references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md), [Array], [RingBuffer], [Buffer].
- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and
loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
# Expressions
@@ -68,6 +66,26 @@ You can use blocks in if statement, as expected:
limited
>>> 120.0
## Enums (quick intro)
Lyng supports simple enums for a fixed set of named constants. Declare with `enum Name { ... }` and use entries as `Name.ENTRY`.
enum Color {
RED, GREEN, BLUE
}
assert( Color.RED is Color )
assertEquals( 0, Color.RED.ordinal )
assertEquals( "BLUE", Color.BLUE.name )
// All entries as a list, in order:
assertEquals( [Color.RED, Color.GREEN, Color.BLUE], Color.entries )
// Lookup by name:
assertEquals( Color.GREEN, Color.valueOf("GREEN") )
For more details (usage patterns, `when` switching, serialization), see OOP notes: [Enums in detail](OOP.md#enums).
When putting multiple statments in the same line it is convenient and recommended to use `;`:
var from; var to
@@ -171,7 +189,8 @@ allow to improve code look and readability. There are borrowed from Kotlin:
### let
`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's returned value. It is useful dealing with null or to
`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's
returned value. It is useful dealing with null or to
get a snapshot of some externally varying value, or with `?.` to process nullable value in a safe manner:
// this state is changed from parallel processes
@@ -206,7 +225,7 @@ Much like let, but it does not alter returned value:
>>> void
While it is not altering return value, the source object could be changed:
also
class Point(x,y)
val p = Point(1,2).also { it.x++ }
assertEquals(p.x, 2)
@@ -222,6 +241,19 @@ It works much like `also`, but is executed in the context of the source object:
assertEquals(p, Point(2,3))
>>> void
## run
Executes a block after it returning the value passed by the block. for example, can be used with elvis operator:
var someVar = null
val result = someVar ?: run {
someVar = 121
"reset"
}
assertEquals("reset", result)
assertEquals(121, someVar)
>>> void
## Math
It is rather simple, like everywhere else:
@@ -258,6 +290,8 @@ Logical operation could be used the same
| === | | Any | (2) |
| !== | | Any | (2) |
| != | | Any | (1) |
| =~ | | | (3) |
| !~ | | | (3) |
| ++a, a++ | | Int | |
| --a, a-- | | Int | |
@@ -269,6 +303,9 @@ Logical operation could be used the same
singleton object, like `null`, are referentially equal too, while string different literals even being equal are most
likely referentially not equal
(3)
: Implemented now in String and Regex as regular expression match and not match, see [Regex].
Reference quality and object equality example:
assert( null == null) // singletons
@@ -465,7 +502,6 @@ after function call, it is treated as a last argument to the call, e.g.:
assert( [11, 21, 31] == mapped)
>>> void
# Lists (aka arrays)
Lyng has built-in mutable array class `List` with simple literals:
@@ -473,7 +509,8 @@ Lyng has built-in mutable array class `List` with simple literals:
[1, "two", 3.33].size
>>> 3
[List] is an implementation of the type `Array`, and through it `Collection` and [Iterable].
[List] is an implementation of the type `Array`, and through it `Collection` and [Iterable]. Please read [Iterable],
many collection based methods are implemented there.
Lists can contain any type of objects, lists too:
@@ -490,21 +527,21 @@ Notice usage of indexing. You can use negative indexes to offset from the end of
When you want to "flatten" it to single array, you can use splat syntax:
[1, ...[2,3], 4]
>>> [1, 2, 3, 4]
>>> [1,2,3,4]
Of course, you can splat from anything that is List (or list-like, but it will be defined later):
val a = ["one", "two"]
val b = [10.1, 20.2]
["start", ...b, ...a, "end"]
>>> ["start", 10.1, 20.2, "one", "two", "end"]
>>> [start,10.1,20.2,one,two,end]
Of course, you can set any list element:
val a = [1, 2, 3]
a[1] = 200
a
>>> [1, 200, 3]
>>> [1,200,3]
Lists are comparable, and it works well as long as their respective elements are:
@@ -535,7 +572,8 @@ The simplest way to concatenate lists is using `+` and `+=`:
void
>>> void
***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will always add all its contents. Use `list.add` to add a single iterable instance:
***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will
always add all its contents. Use `list.add` to add a single iterable instance:
var list = [1, 2]
val other = [3, 4]
@@ -563,7 +601,6 @@ Use `list.add` to avoid confusion:
assert( list == [1, 2, [3, 4], (10..12)])
>>> void
To add elements to the list:
val x = [1,2]
@@ -594,21 +631,20 @@ Using splat arguments can simplify inserting list in list:
val x = [1, 2, 3]
x.insertAt( 1, ...[0,100,0])
x
>>> [1, 0, 100, 0, 2, 3]
>>> [1,0,100,0,2,3]
Note that to add to the end you still need to use `add` or positive index of the after-last element:
val x = [1,2,3]
x.insertAt(3, 10)
x
>>> [1, 2, 3, 10]
>>> [1,2,3,10]
but it is much simpler, and we recommend to use '+='
val x = [1,2,3]
x += 10
>>> [1, 2, 3, 10]
>>> [1,2,3,10]
## Removing list items
@@ -660,14 +696,63 @@ Please see [Set] for detailed description.
# Maps
Maps are unordered collection of key-value pairs, where keys are unique. See [Map] for details. Map also
are [Iterable]:
Maps are unordered collections of key-value pairs, where keys are unique. Maps are also [Iterable].
val m = Map( "foo" => 77, "bar" => "buzz" )
assertEquals( m["foo"], 77 )
You can create them either with the classic constructor (still supported):
val old = Map( "foo" => 77, "bar" => "buzz" )
assertEquals( old["foo"], 77 )
>>> void
Please see [Map] reference for detailed description on using Maps.
…or with map literals, which are often more convenient:
val x = 10
val y = 10
val m = { "a": 1, x: x * 2, y: }
// identifier keys become strings; `y:` is shorthand for y: y
assertEquals(1, m["a"]) // string-literal key
assertEquals(20, m["x"]) // identifier key
assertEquals(10, m["y"]) // shorthand
>>> void
Map literals support trailing commas for nicer diffs:
val m2 = {
"a": 1,
b: 2,
}
assertEquals(1, m2["a"])
assertEquals(2, m2["b"])
>>> void
You can spread other maps inside a literal with `...`. Items merge left-to-right and the rightmost value wins:
val base = { a: 1, b: 2 }
val merged = { a: 0, ...base, b: 3, c: 4 }
assertEquals(1, merged["a"]) // base overwrites a:0
assertEquals(3, merged["b"]) // literal overwrites spread
assertEquals(4, merged["c"]) // new key
>>> void
Merging also works with `+` and `+=`, and you can combine maps and entries conveniently:
val m3 = { "a": 10 } + ("b" => 20)
assertEquals(10, m3["a"])
assertEquals(20, m3["b"])
var m4 = ("x" => 1) + ("y" => 2) // entry + entry → Map
m4 += { z: 3 } // merge literal
assertEquals(1, m4["x"])
assertEquals(2, m4["y"])
assertEquals(3, m4["z"])
>>> void
Notes:
- Access keys with brackets: `m["key"]`. There is no `m.key` sugar.
- Empty `{}` remains a block/lambda; use `Map()` to create an empty map.
- When you need computed (expression) keys or non-string keys, use `Map(...)` constructor with entries, e.g. `Map( ("a" + "b") => 1 )`, then merge with a literal if needed: `{ base: } + (computedKey => 42)`.
Please see the [Map] reference for a deeper guide.
# Flow control operators
@@ -696,6 +781,8 @@ Or, more neat:
## When
See also: [Comprehensive guide to `when`](when.md)
It is very much like the kotlin's:
fun type(x) {
@@ -742,23 +829,28 @@ Also, you can check the type too:
#### Contains:
You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and `!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be thrown.
You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and
`!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be
thrown.
Typical builtin types that are containers (e.g. support `conain`):
Typical builtin types that are containers (e.g. support `contains`):
| class | notes |
|------------|------------------------------------------------|
| Collection | contains an element (1) |
| Array | faster maybe that Collection's |
| List | faster than Array's |
| String | character in string or substring in string (3) |
| Range | object is included in the range (2) |
| class | notes |
|--------------|------------------------------------------------|
| [Collection] | contains an element (1) |
| Array | faster maybe that Collection's |
| List | faster than Array's |
| String | character in string or substring in string (3) |
| Range | object is included in the range (2) |
| Buffer | byte is in buffer |
| RingBuffer | object is in buffer |
(1)
: Iterable is not the container as it can be infinite
(2)
: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is usually not what you expect of it:
: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is
usually not what you expect of it:
assert( "more" in "a".."z") // string range ok
assert( 'x' !in "a".."z") // char in string range: probably error
@@ -767,7 +859,9 @@ Typical builtin types that are containers (e.g. support `conain`):
>>> void
(3)
: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array]. String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it it would be iterable).
: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array].
String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it
it would be iterable).
So we recommend not to mix characters and string ranges; use `ch in str` that works
as expected:
@@ -845,7 +939,8 @@ We can skip the rest of the loop and restart it, as usual, with `continue` opera
"found even numbers: " + countEven
>>> "found even numbers: 5"
`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we intentionally avoid using for loops here):
`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we
intentionally avoid using for loops here):
var count = 0
var total = 0
@@ -891,21 +986,22 @@ test function (remember function return it's last expression result):
```mermaid
flowchart TD
S((start)) --> Cond{check}
Cond--false, no else--->V((void))
Cond--true-->E(["last = loop_body()" ])
E--break value---->BV((value))
E--> Check2{check}
E--break---->V
Check2 --false-->E
Check2 --true, no else--->L((last))
Check2 --true, else-->Else(["else_clause()"])
Cond--false, else--->Else
Cond -- false, no else ---> V((void))
Cond -- true --> E(["last = loop_body()"])
E -- break value ----> BV((value))
E --> Check2{check}
E -- break ----> V
Check2 -- false --> E
Check2 -- true, no else ---> L((last))
Check2 -- true, else --> Else(["else_clause()"])
Cond -- false, else ---> Else
Else --> Ele4$nr((else))
```
So the returned value, as seen from diagram could be one of:
- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the empty break was executed.
- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the
empty break was executed.
- value returned from `break value' statement
- value returned from the `else` clause, of the loop was not broken
- value returned from the last execution of loop body, if there was no `break` and no `else` clause.
@@ -929,7 +1025,8 @@ available in the condition:
} while( continueLoop )
>>> "OK"
This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value returning by the loop:
This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value
returning by the loop:
fun readLine() { "done: result" }
val result = do {
@@ -940,7 +1037,6 @@ This is sometimes convenient when condition is complex and has to be calculated
Suppose readLine() here reads some stream of lines.
## For loops
For loop are intended to traverse collections, and all other objects that supports
@@ -983,7 +1079,8 @@ We can use labels too:
# Exception handling
Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the catch block that caught the exception:
Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the
catch block that caught the exception:
var error = "not caught"
var finallyCaught = false
@@ -1036,7 +1133,6 @@ many more.
- see [exception handling](exceptions_handling.md) for detailed exceptions tutorial and reference.
# Self-assignments in expression
There are auto-increments and auto-decrements:
@@ -1112,6 +1208,27 @@ These should be imported from [lyng.time](time.md). For example:
See [more docs on time manipulation](time.md)
# Enums
For the moment, only simple enums are implemented. Enum is a list of constants, represented also by their
_ordinal_ - [Int] value.
enum Color {
RED, GREEN, BLUE
}
assert( Color.RED is Color )
assertEquals( 2, Color.BLUE.ordinal )
assertEquals( "BLUE", Color.BLUE.name )
assertEquals( [Color.RED,Color.GREEN,Color.BLUE], Color.entries)
assertEquals( Color.valueOf("GREEN"), Color.GREEN )
>>> void
Enums are serialized as ordinals. Please note that due to caching, serialized string arrays could be even more compact
than enum arrays, until `Lynon.encodeTyped` will be implemented.
# Comments
// single line comment
@@ -1138,16 +1255,41 @@ See also [math operations](math.md)
The type for the character objects is `Char`.
### String literal escapes
| escape | ASCII value |
|--------|-----------------------|
| \n | 0x10, newline |
| \r | 0x13, carriage return |
| \t | 0x07, tabulation |
| \\ | \ slash character |
| \" | " double quote |
Other `\c` combinations, where c is any char except mentioned above, are left intact, e.g.:
val s = "\a"
assert(s[0] == '\')
assert(s[1] == 'a')
>>> void
same as:
val s = "\\a"
assert(s[0] == '\')
assert(s[1] == 'a')
>>> void
### Char literal escapes
Are the same as in string literals with little difference:
| escape | ASCII value |
|--------|-------------------|
| \n | 0x10, newline |
| \t | 0x07, tabulation |
| \\ | \ slash character |
| \' | ' apostrophe |
| escape | ASCII value |
|--------|-----------------------|
| \n | 0x10, newline |
| \r | 0x13, carriage return |
| \t | 0x07, tabulation |
| \\ | \ slash character |
| \' | ' apostrophe |
### Char instance members
@@ -1159,7 +1301,6 @@ Are the same as in string literals with little difference:
| code | Int | Unicode code for the character |
| | | |
## String details
Strings are arrays of Unicode characters. It can be indexed, and indexing will
@@ -1187,7 +1328,10 @@ To format a string use sprintf-style modifiers like:
assertEquals( "hello :11 ", "%-6s:%-6d"(a, b) )
>>> void
List of format specifiers closely resembles C sprintf() one. See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
List of format specifiers closely resembles C sprintf() one.
See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe
using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types
are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
This list will be extended.
@@ -1214,36 +1358,57 @@ Open-ended ranges could be used to get start and end too:
### String operations
Concatenation is a `+`: `"hello " + name` works as expected. No confusion.
Concatenation is a `+`: `"hello " + name` works as expected. No confusion. There is also
[Regex] support for strings, see the link, for example, whole string match:
assert( !"123".matches( "\d\d".re ) )
assert( "123".matches( "\d\d\d".re ) )
>>> void
Extraction:
"abcd42def"[ "\d+".re ].value
>>> "42"
Part match:
assert( "abc foo def" =~ "f[oO]+".re )
assert( "foo" == $~.value )
>>> void
Typical set of String functions includes:
| fun/prop | description / notes |
|-------------------|------------------------------------------------------------|
| lower() | change case to unicode upper |
| upper() | change case to unicode lower |
| fun/prop | description / notes |
|--------------------|------------------------------------------------------------|
| lower() | change case to unicode upper |
| upper() | change case to unicode lower |
| trim() | trim space chars from both ends |
| startsWith(prefix) | true if starts with a prefix |
| endsWith(prefix) | true if ends with a prefix |
| take(n) | get a new string from up to n first characters |
| takeLast(n) | get a new string from up to n last characters |
| drop(n) | get a new string dropping n first chars, or empty string |
| dropLast(n) | get a new string dropping n last chars, or empty string |
| size | size in characters like `length` because String is [Array] |
| (args...) | sprintf-like formatting, see [string formatting] |
| [index] | character at index |
| [Range] | substring at range |
| s1 + s2 | concatenation |
| s1 += s2 | self-modifying concatenation |
| toReal() | attempts to parse string as a Real value |
| toInt() | parse string to Int value |
| characters() | create [List] of characters (1) |
| encodeUtf8() | returns [Buffer] with characters encoded to utf8 |
| endsWith(prefix) | true if ends with a prefix |
| last() | get last character of a string or throw |
| take(n) | get a new string from up to n first characters |
| takeLast(n) | get a new string from up to n last characters |
| drop(n) | get a new string dropping n first chars, or empty string |
| dropLast(n) | get a new string dropping n last chars, or empty string |
| size | size in characters like `length` because String is [Array] |
| (args...) | sprintf-like formatting, see [string formatting] |
| [index] | character at index |
| [Range] | substring at range (2) |
| [Regex] | find first match of regex, like [Regex.find] (2) |
| s1 + s2 | concatenation |
| s1 += s2 | self-modifying concatenation |
| toReal() | attempts to parse string as a Real value |
| toInt() | parse string to Int value |
| characters() | create [List] of characters (1) |
| encodeUtf8() | returns [Buffer] with characters encoded to utf8 |
| matches(re) | matches the regular expression (2) |
| | |
(1)
: List is mutable therefore a new copy is created on each call.
(2)
: See [Regex]
### Literals
@@ -1252,16 +1417,45 @@ String literal could be multiline:
"Hello
World"
though multiline literals is yet work in progress.
In this case, it will be passed literally ot "hello\n World". But, if there are
several lines with common left indent, it will be removed, also, forst and last lines,
if blank, will be removed too, for example:
println("
This is a multiline text.
This is a second line.
")
>>> This is a multiline text.
>>> This is a second line.
>>> void
- as expected, empty lines and common indent were removed. It is much like kotlin's `""" ... """.trimIndent()`
technique, but simpler ;)
# Built-in functions
See [math functions](math.md). Other general purpose functions are:
| name | description |
|----------------------------------------------|----------------------------------------------------------|
| assert(condition,message="assertion failed") | runtime code check. There will be an option to skip them |
| println(args...) | Open for overriding, it prints to stdout. |
| name | description |
|---------------------------------------|------------------------------------------------------------|
| assert(condition, fn) | (1) runtime code check with generic or custom nessage `fn` |
| assertEquals(a,b) | |
| assertNotEquals(a,b) | |
| assertTrows { /* block */ } | |
| check(condition, message=<default>) | throws IllegalStateException" of condition isn't met |
| require(condition, message=<default>) | throws IllegalArgumentException" of condition isn't met |
| println(args...) | Open for overriding, it prints to stdout with newline. |
| print(args...) | Open for overriding, it prints to stdout without newline. |
| flow {} | create flow sequence, see [parallelism] |
| delay, launch, yield | see [parallelism] |
| cached(builder) | remembers builder() on first invocation and return it then |
| let, also, apply, run | see above, flow controls |
(1)
: `fn` is optional lambda returning string message to add to exception string.
Lambda avoid unnecessary execution if assertion is not failed. for example:
assert( x < 10 ) { "x=%s should be < 10"(x) }
# Built-in constants
@@ -1271,13 +1465,96 @@ See [math functions](math.md). Other general purpose functions are:
| π | See [math](math.md) |
[List]: List.md
[Iterable]: Iterable.md
[Iterator]: Iterator.md
[Real]: Real.md
[Range]: Range.md
[String]: String.md
[String]: development/String.md
[string formatting]: https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary
[Set]: Set.md
[Map]: Map.md
[Buffer]: Buffer.md
[parallelism]: parallelism.md
[RingBuffer]: RingBuffer.md
[Collection]: Collection.md
[Array]: Array.md
[Regex]: Regex.md
## Multiple Inheritance (quick start)
Lyng supports multiple inheritance (MI) with simple, predictable rules. For a full reference see OOP notes, this is a quick, copy‑paste friendly overview.
Declare a class with multiple bases and pass constructor arguments to each base in the header:
```
class Foo(val a) {
var tag = "F"
fun runA() { "ResultA:" + a }
fun common() { "CommonA" }
private fun privateInFoo() {}
protected fun protectedInFoo() {}
}
class Bar(val b) {
var tag = "B"
fun runB() { "ResultB:" + b }
fun common() { "CommonB" }
}
class FooBar(a, b) : Foo(a), Bar(b) {
// Inside class bodies you can qualify with this@Type
fun fromFoo() { this@Foo.common() }
fun fromBar() { this@Bar.common() }
}
val fb = FooBar(1, 2)
assertEquals("ResultA:1", fb.runA())
assertEquals("ResultB:2", fb.runB())
// Unqualified ambiguous member uses the first base (leftmost)
assertEquals("CommonA", fb.common())
// Disambiguate with casts
assertEquals("CommonB", (fb as Bar).common())
assertEquals("CommonA", (fb as Foo).common())
// Field collisions: unqualified read/write uses the first in order
assertEquals("F", fb.tag)
fb.tag = "X"
assertEquals("X", fb.tag) // Foo.tag updated
assertEquals("X", (fb as Foo).tag) // qualified read: Foo.tag
assertEquals("B", (fb as Bar).tag) // qualified read: Bar.tag
// Qualified write targets the chosen base storage
(fb as Bar).tag = "Y"
assertEquals("X", (fb as Foo).tag)
assertEquals("Y", (fb as Bar).tag)
// Optional casts with safe call
class Buzz : Bar(3)
val buzz = Buzz()
assertEquals("ResultB:3", buzz.runB())
assertEquals("ResultB:3", (buzz as? Bar)?.runB())
assertEquals(null, (buzz as? Foo)?.runA())
```
Notes:
- Resolution order uses C3 MRO (active): deterministic, monotonic order suitable for diamonds and complex hierarchies. Example: for `class D() : B(), C()` where both `B()` and `C()` derive from `A()`, the C3 order is `D → B → C → A`. The first visible match wins.
- `private` is visible only inside the declaring class; `protected` is visible from the declaring class and any of its transitive subclasses. Qualialsofication (`this@Type`) or casts do not bypass visibility.
- Safe‑call `?.` works with `as?` for optional dispatch.
To get details on OOP in Lyng, see [OOP notes](oop.md).
+241
View File
@@ -0,0 +1,241 @@
# The `when` statement (expression)
[//]: # (topMenu)
Lyng provides a concise multi-branch selection with `when`, heavily inspired by Kotlin. In Lyng, `when` is an expression: it evaluates to a value. If the selected branch contains no value (e.g., it ends with `void` or calls a void function like `println`), the whole `when` expression evaluates to `void`.
Currently, Lyng implements the "subject" form: `when(value) { ... }`. The subject-less form `when { condition -> ... }` is not implemented yet.
## Quick examples
val r1 = when("a") {
"a" -> "ok"
else -> "nope"
}
assertEquals("ok", r1)
val r2 = when(5) {
3 -> "no"
4 -> "no"
else -> "five"
}
assertEquals("five", r2)
val r3 = when(5) {
3 -> "no"
4 -> "no"
}
// no matching case and no else → `void`
assert(r3 == void)
>>> void
## Syntax
when(subject) {
condition1 [, condition2, ...] -> resultExpressionOrBlock
conditionN -> result
else -> fallback
}
- Commas group multiple conditions for one branch.
- First matching branch wins; there is no fall‑through.
- `else` is optional. If omitted and nothing matches, the result is `void`.
## Matching rules (conditions)
Within `when(subject)`, each condition is evaluated against the already evaluated `subject`. Lyng supports:
1) Equality match (default)
- Any expression value can be used as a condition. It matches if it is equal to `subject`.
- Equality relies on Lyng’s comparison (`compareTo(...) == 0`). For user types, implement comparison accordingly.
when(x) {
0 -> "zero"
"EUR" -> "currency"
}
>>> void
2) Type checks: `is` and `!is`
- Check whether the subject is an instance of a class.
- Works with built‑in classes and user classes.
fun typeOf(x) {
when(x) {
is Real, is Int -> "number"
is String -> "string"
else -> "other"
}
}
assertEquals("number", typeOf(5))
assertEquals("string", typeOf("hi"))
>>> void
3) Containment checks: `in` and `!in`
- `in container` matches if `container.contains(subject)` is true.
- `!in container` matches if `contains(subject)` is false.
- Any object that provides `contains(item)` can act as a container.
Common containers:
- Ranges (e.g., `'a'..'z'`, `1..10`, `1..<10`, `..5`, `5..`)
- Lists, Sets, Arrays, Buffers
- Strings (character or substring containment)
Examples:
when('e') {
in 'a'..'c' -> "small"
in 'a'..'z' -> "letter"
else -> "other"
}
>>> "letter"
when(5) {
in [1,2,3,4,6] -> "no"
in [7,0,9] -> "no"
else -> "ok"
}
>>> "ok"
when(5) {
in [1,2,3,4,6] -> "no"
in [7,0,9] -> "no"
in [-1,5,11] -> "yes"
else -> "no"
}
>>> "yes"
when(5) {
!in [1,2,3,4,6,5] -> "no"
!in [7,0,9,5] -> "no"
!in [-1,15,11] -> "ok"
else -> "no"
}
>>> "ok"
// String containment
"foo" in "foobar" // true (substring)
'o' in "foobar" // true (character)
>>> true
Notes on mixed String/Char ranges:
- Prefer character ranges for characters: `'a'..'z'`.
- `"a".."z"` is a String range and may not behave as you expect with `Char` subjects.
assert( "more" in "a".."z")
assert( 'x' !in "a".."z") // Char vs String range: often not what you want
assert( 'x' in 'a'..'z') // OK
assert( "x" !in 'a'..'z') // String in Char range: likely not intended
>>> void
## Grouping multiple conditions with commas
You can group values and/or `is`/`in` checks for a single result:
fun classify(x) {
when(x) {
"42", 42 -> "answer"
is Real, is Int -> "number"
in ['@', '#', '^'] -> "punct1"
in "*&.," -> "punct2"
else -> "unknown"
}
}
assertEquals("number", classify(π/2))
assertEquals("answer", classify(42))
assertEquals("answer", classify("42"))
>>> void
## Return value and blocks
- `when` returns the value of the matched branch result expression/block.
- Branch bodies can be single expressions or blocks `{ ... }`.
- If a matched branch produces `void` (e.g., only prints), the `when` result is `void`.
val res = when(2) {
1 -> 10
2 -> { println("two"); 20 }
else -> 0
}
assertEquals(20, res)
>>> void
## Else branch
- Optional but recommended when non‑exhaustive.
- If omitted and nothing matches, `when` result is `void` (see r3 in the Quick examples).
- Only one `else` is allowed.
## Subject‑less `when`
The Kotlin‑style subject‑less form `when { condition -> ... }` is not implemented yet in Lyng. Use `if/else` chains or structure your checks around a subject with `when(subject) { ... }`.
## Extending `when` for your own types
### Equality matches
- Equality checks in `when(subject)` use Lyng comparison (`compareTo` semantics under the hood). For your own Lyng classes, implement comparison appropriately so that `subject == value` works as intended.
### `in` / `!in` containment
- Provide a `contains(item)` method on your class to participate in `in` conditions.
- Example: a custom `Box` that contains one specific item:
class Box(val item)
fun Box.contains(x) { x == item }
val b = Box(10)
when(10) { in b -> "hit" }
>>> "hit"
Any built‑in collection (`List`, `Set`, `Array`), `Range`, `Buffer`, and other containers already implement `contains`.
### Type checks (`is` / `!is`)
- Every value has a `::class` that yields its Lyng class object, e.g. `[1,2,3]::class` → `List`.
- `is ClassName` in `when` uses Lyng’s class hierarchy. Ensure your class is declared and can be referenced by name.
[]::class == List
>>> true
fun f(x) { when(x) { is List -> "list" else -> "other" } }
assertEquals("list", f([1]))
>>> void
## Kotlin‑backed classes (embedding)
When embedding Lyng in Kotlin, you may expose Kotlin‑backed objects and classes. Interactions inside `when` work as follows:
- `is` checks use the Lyng class object you expose for your Kotlin type. Ensure your exposed class participates in the Lyng class hierarchy (see Embedding docs).
- `in` checks call `contains(subject)`; if your Kotlin‑backed object wants to support `in`, expose a `contains(item)` method (mapped to Lyng) or implement the corresponding Lyng container wrapper.
- Equality follows Lyng comparison rules. Ensure your Kotlin‑backed object’s Lyng adapter implements equality/compare correctly.
For details on exposing classes/methods from Kotlin, see: [Embedding Lyng in your Kotlin project](embedding.md).
## Gotchas and tips
- First match wins; there is no fall‑through. Order branches carefully.
- Group related conditions with commas for readability and performance (a single branch evaluation).
- Prefer character ranges for character tests; avoid mixing `String` and `Char` ranges.
- If you rely on `in`, check that your container implements `contains(item)`.
- Remember: `when` is an expression — you can assign its result to a variable or return it from a function.
## Additional examples
fun label(ch) {
when(ch) {
in '0'..'9' -> "digit"
in 'a'..'z', in 'A'..'Z' -> "letter"
'$' -> "dollar"
else -> "other"
}
}
assertEquals("digit", label('3'))
assertEquals("dollar", label('$'))
>>> void
fun normalize(x) {
when(x) {
is Int -> x
is Real -> x.round()
else -> 0
}
}
assertEquals(12, normalize(12))
assertEquals(3, normalize(2.6))
>>> void
+68
View File
@@ -0,0 +1,68 @@
Lyng TextMate grammar
======================
This folder contains a TextMate grammar for the Lyng language so you can get syntax highlighting quickly in:
- JetBrains IDEs (IntelliJ IDEA, Fleet, etc.) via “TextMate Bundles”
- VS Code (via “Install from VSIX” or by adding as an extension folder)
Files
-----
- `package.json` — VS Code–style wrapper that JetBrains IDEs can import as a TextMate bundle.
- `syntaxes/lyng.tmLanguage.json` — the grammar. It highlights:
- Line and block comments (`//`, `/* */`)
- Shebang line at file start (`#!...`)
- Strings: single and double quotes with escapes
- Char literals `'x'` with escapes
- Numbers: decimal with underscores and exponents, and hex (`0x...`)
- Keywords (control and declarations), boolean operator words (`and`, `or`, `not`, `in`, `is`, `as`, `as?`)
- Composite textual operators: `not in`, `not is`
- Constants: `true`, `false`, `null`, `this`
- Annotations: `@name` (Unicode identifiers supported)
- Labels: `name:` (Unicode identifiers supported)
- Declarations: highlights declared names in `fun|fn name`, `class|enum Name`, `val|var name`
- Types: built-ins (`Int|Real|String|Bool|Char|Regex`) and Capitalized identifiers (heuristic)
- Operators including ranges (`..`, `..<`, `...`), null-safe (`?.`, `?[`, `?(`, `?{`, `?:`, `??`), arrows (`->`, `=>`, `::`), match operators (`=~`, `!~`), bitwise, arithmetic, etc.
- Shuttle operator `<=>`
- Division operator `/` (note: Lyng has no regex literal syntax; `/` is always division)
- Named arguments at call sites `name: value` (the `name` part is highlighted as `variable.parameter.named.lyng` and the `:` as punctuation). The rule is anchored to `(` or `,` and excludes `::` to avoid conflicts.
Install in IntelliJ IDEA (and other JetBrains IDEs)
---------------------------------------------------
1. Open Settings / Preferences → Editor → TextMate Bundles.
2. Click “+” and select this folder `editors/lyng-textmate/` (the folder that contains `package.json`).
3. Ensure `*.lyng` is associated with the Lyng grammar (IntelliJ usually picks this up from `fileTypes`).
4. Optional: customize colors with Settings → Editor → Color Scheme → TextMate.
Enable Markdown code-fence highlighting in IntelliJ
--------------------------------------------------
1. Settings / Preferences → Languages & Frameworks → Markdown → Code style → Code fences → Languages.
2. Add mapping: language id `lyng` → “Lyng (TextMate)”.
3. Now blocks like
```
```lyng
// Lyng code here
```
```
will be highlighted.
Install in VS Code
------------------
Fastest local install without packaging:
1. Copy or symlink this folder somewhere stable (or keep it in your workspace).
2. Use “Developer: Install Extension from Location…” (Insiders) or package with `vsce package` and install the resulting `.vsix`.
3. VS Code will auto-associate `*.lyng` via this extension; if needed, check File Associations.
Notes and limitations
---------------------
- Type highlighting is heuristic (Capitalized identifiers). The IntelliJ plugin will use language semantics and avoid false positives.
- If your language adds or changes tokens, please update patterns in `lyng.tmLanguage.json`. The Kotlin sources in `lynglib/src/commonMain/kotlin/net/sergeych/lyng/highlight/` are a good reference for token kinds.
- Labels `name:` at statement level remain supported and are kept distinct from named call arguments by context. The grammar prefers named-argument matching when a `name:` appears right after `(` or `,`.
Lyng specifics
--------------
- There are no regex literal tokens in Lyng at the moment; the slash character `/` is always treated as the division operator. The grammar intentionally does not define a `/.../` regex rule to avoid mis-highlighting lines like `a / b`.
Contributing
------------
Pull requests to refine patterns and add tests/samples are welcome. You can place test snippets in `sample_texts/` and visually verify.
+25
View File
@@ -0,0 +1,25 @@
{
"name": "lyng-textmate",
"displayName": "Lyng",
"description": "TextMate grammar for the Lyng language (for JetBrains IDEs via TextMate Bundles and VS Code).",
"version": "0.0.3",
"publisher": "lyng",
"license": "Apache-2.0",
"engines": { "vscode": "^1.0.0" },
"contributes": {
"languages": [
{
"id": "lyng",
"aliases": ["Lyng", "lyng"],
"extensions": [".lyng"]
}
],
"grammars": [
{
"language": "lyng",
"scopeName": "source.lyng",
"path": "./syntaxes/lyng.tmLanguage.json"
}
]
}
}
@@ -0,0 +1,86 @@
{
"$schema": "https://raw.githubusercontent.com/martinring/tmlanguage/master/tmlanguage.json",
"name": "Lyng",
"scopeName": "source.lyng",
"fileTypes": ["lyng"],
"patterns": [
{ "include": "#shebang" },
{ "include": "#comments" },
{ "include": "#strings" },
{ "include": "#char" },
{ "include": "#numbers" },
{ "include": "#declarations" },
{ "include": "#keywords" },
{ "include": "#constants" },
{ "include": "#types" },
{ "include": "#mapLiterals" },
{ "include": "#namedArgs" },
{ "include": "#annotations" },
{ "include": "#labels" },
{ "include": "#directives" },
{ "include": "#operators" },
{ "include": "#punctuation" }
],
"repository": {
"shebang": { "patterns": [ { "name": "comment.line.shebang.lyng", "match": "^#!.*$" } ] },
"comments": {
"patterns": [
{ "name": "comment.line.double-slash.lyng", "match": "//.*$" },
{ "name": "comment.block.lyng", "begin": "/\\*", "end": "\\*/" }
]
},
"strings": {
"patterns": [
{ "name": "string.quoted.double.lyng", "begin": "\"", "end": "\"", "patterns": [ { "match": "\\\\.", "name": "constant.character.escape.lyng" } ] },
{ "name": "string.quoted.single.lyng", "begin": "'", "end": "'", "patterns": [ { "match": "\\\\.", "name": "constant.character.escape.lyng" } ] }
]
},
"char": { "patterns": [ { "name": "constant.character.lyng", "match": "'(?:[^\\\\']|\\\\.)'" } ] },
"numbers": {
"patterns": [
{ "name": "constant.numeric.hex.lyng", "match": "0x[0-9A-Fa-f_]+" },
{ "name": "constant.numeric.decimal.lyng", "match": "(?<![A-Za-z_])(?:[0-9][0-9_]*)\\.(?:[0-9_]+)(?:[eE][+-]?[0-9_]+)?|(?<![A-Za-z_])(?:[0-9][0-9_]*)(?:[eE][+-]?[0-9_]+)?" }
]
},
"annotations": { "patterns": [ { "name": "entity.name.label.at.lyng", "match": "@[\\p{L}_][\\p{L}\\p{N}_]*:" }, { "name": "storage.modifier.annotation.lyng", "match": "@[\\p{L}_][\\p{L}\\p{N}_]*" } ] },
"mapLiterals": {
"patterns": [
{
"name": "meta.map.entry.lyng",
"match": "(?:(?<=\\{|,))\\s*(\"[^\"]*\"|[\\p{L}_][\\p{L}\\p{N}_]*)\\s*(:)(?!:)",
"captures": {
"1": { "name": "variable.other.property.key.lyng" },
"2": { "name": "punctuation.separator.colon.lyng" }
}
},
{
"name": "meta.map.spread.lyng",
"match": "(?:(?<=\\{|,))\\s*(\\.\\.\\.)",
"captures": {
"1": { "name": "keyword.operator.spread.lyng" }
}
}
]
},
"namedArgs": {
"patterns": [
{
"name": "meta.argument.named.lyng",
"match": "(?:(?<=\\()|(?<=,))\\s*([\\p{L}_][\\p{L}\\p{N}_]*)\\s*(:)(?!:)",
"captures": {
"1": { "name": "variable.parameter.named.lyng" },
"2": { "name": "punctuation.separator.colon.lyng" }
}
}
]
},
"labels": { "patterns": [ { "name": "entity.name.label.lyng", "match": "[\\p{L}_][\\p{L}\\p{N}_]*:" } ] },
"directives": { "patterns": [ { "name": "meta.directive.lyng", "match": "^\\s*#[_A-Za-z][_A-Za-z0-9]*" } ] },
"declarations": { "patterns": [ { "name": "meta.function.declaration.lyng", "match": "\\b(?:fun|fn)\\s+([\\p{L}_][\\p{L}\\p{N}_]*)", "captures": { "1": { "name": "entity.name.function.lyng" } } }, { "name": "meta.type.declaration.lyng", "match": "\\b(?:class|enum)\\s+([\\p{L}_][\\p{L}\\p{N}_]*)", "captures": { "1": { "name": "entity.name.type.lyng" } } }, { "name": "meta.variable.declaration.lyng", "match": "\\b(?:val|var)\\s+([\\p{L}_][\\p{L}\\p{N}_]*)", "captures": { "1": { "name": "variable.other.declaration.lyng" } } } ] },
"keywords": { "patterns": [ { "name": "keyword.control.lyng", "match": "\\b(?:if|else|when|while|do|for|try|catch|finally|throw|return|break|continue)\\b" }, { "name": "keyword.declaration.lyng", "match": "\\b(?:fun|fn|class|enum|val|var|import|package|constructor|property|open|extern|private|protected|static)\\b" }, { "name": "keyword.operator.word.lyng", "match": "\\bnot\\s+(?:in|is)\\b" }, { "name": "keyword.operator.word.lyng", "match": "\\b(?:and|or|not|in|is|as|as\\?)\\b" } ] },
"constants": { "patterns": [ { "name": "constant.language.lyng", "match": "(?:\\b(?:true|false|null|this)\\b|π)" } ] },
"types": { "patterns": [ { "name": "storage.type.lyng", "match": "\\b(?:Int|Real|String|Bool|Char|Regex)\\b" }, { "name": "entity.name.type.lyng", "match": "\\b[A-Z][A-Za-z0-9_]*\\b(?!\\s*\\()" } ] },
"operators": { "patterns": [ { "name": "keyword.operator.comparison.lyng", "match": "===|!==|==|!=|<=|>=|<|>" }, { "name": "keyword.operator.shuttle.lyng", "match": "<=>" }, { "name": "keyword.operator.arrow.lyng", "match": "=>|->|::" }, { "name": "keyword.operator.range.lyng", "match": "\\.\\.\\.|\\.\\.<|\\.\\." }, { "name": "keyword.operator.nullsafe.lyng", "match": "\\?\\.|\\?\\[|\\?\\(|\\?\\{|\\?:|\\?\\?" }, { "name": "keyword.operator.assignment.lyng", "match": "(?:\\+=|-=|\\*=|/=|%=|=)" }, { "name": "keyword.operator.logical.lyng", "match": "&&|\\|\\|" }, { "name": "keyword.operator.bitwise.lyng", "match": "<<|>>|&|\\||\\^|~" }, { "name": "keyword.operator.match.lyng", "match": "=~|!~" }, { "name": "keyword.operator.arithmetic.lyng", "match": "\\+\\+|--|[+\\-*/%]" }, { "name": "keyword.operator.other.lyng", "match": "[!?]" } ] },
"punctuation": { "patterns": [ { "name": "punctuation.separator.comma.lyng", "match": "," }, { "name": "punctuation.terminator.statement.lyng", "match": ";" }, { "name": "punctuation.section.block.begin.lyng", "match": "[(]{1}|[{]{1}|\\[" }, { "name": "punctuation.section.block.end.lyng", "match": "[)]{1}|[}]{1}|\\]" }, { "name": "punctuation.accessor.dot.lyng", "match": "\\." }, { "name": "punctuation.separator.colon.lyng", "match": ":" } ] }
}
}
+26
View File
@@ -1,3 +1,20 @@
#
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
#Gradle
org.gradle.jvmargs=-Xmx2048M -Dfile.encoding=UTF-8 -Dkotlin.daemon.jvm.options\="-Xmx2048M"
org.gradle.caching=true
@@ -12,3 +29,12 @@ android.nonTransitiveRClass=true
# other
kotlin.native.cacheKind.linuxX64=none
# Workaround: Ensure Gradle uses a JDK with `jlink` available for AGP's JDK image transform.
# On this environment, the system JDK 21 installation lacks `jlink`, causing
# :lynglib:androidJdkImage to fail. Point Gradle to a JDK that includes `jlink`.
# This affects only the JDK Gradle runs with; Kotlin/JVM target remains compatible.
#org.gradle.java.home=/home/sergeych/.jdks/corretto-21.0.9
android.experimental.lint.migrateToK2=false
android.lint.useK2Uast=false
kotlin.mpp.applyDefaultHierarchyTemplate=true
+3 -2
View File
@@ -1,10 +1,10 @@
[versions]
agp = "8.5.2"
clikt = "5.0.3"
kotlin = "2.1.21"
kotlin = "2.2.21"
android-minSdk = "24"
android-compileSdk = "34"
kotlinx-coroutines = "1.9.0"
kotlinx-coroutines = "1.10.2"
mp_bintools = "0.1.12"
firebaseCrashlyticsBuildtools = "3.0.3"
okioVersion = "3.10.2"
@@ -20,6 +20,7 @@ mp_bintools = { module = "net.sergeych:mp_bintools", version.ref = "mp_bintools"
firebase-crashlytics-buildtools = { group = "com.google.firebase", name = "firebase-crashlytics-buildtools", version.ref = "firebaseCrashlyticsBuildtools" }
okio = { module = "com.squareup.okio:okio", version.ref = "okioVersion" }
okio-fakefilesystem = { module = "com.squareup.okio:okio-fakefilesystem", version.ref = "okioVersion" }
okio-nodefilesystem = { module = "com.squareup.okio:okio-nodefilesystem", version.ref = "okioVersion" }
compiler = { group = "androidx.databinding", name = "compiler", version.ref = "compiler" }
[plugins]
+17
View File
@@ -1,3 +1,20 @@
#
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-8.7-bin.zip
Vendored
+3 -2
View File
@@ -1,13 +1,13 @@
#!/bin/sh
#
# Copyright © 2015-2021 the original authors.
# Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# https://www.apache.org/licenses/LICENSE-2.0
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
@@ -15,6 +15,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
#
##############################################################################
#
+32
View File
@@ -0,0 +1,32 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
- Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-->
<svg width="16" height="16" viewBox="0 0 16 16" xmlns="http://www.w3.org/2000/svg" aria-labelledby="title" role="img">
<title>Lyng File Icon (temporary λ)</title>
<defs>
<style>
.g { fill: none; stroke: currentColor; stroke-width: 1.6; stroke-linecap: round; stroke-linejoin: round; }
</style>
</defs>
<!-- Keep shapes crisp on small canvas; slight inset to avoid clipping -->
<g transform="translate(0.5,0.5)">
<!-- Stylized lambda fitted to 14x14 -->
<path class="g" d="M4.5 2.5 L7 9.5 C7.6 11.2 9.0 12.5 11.0 12.5 L14.5 12.5"/>
<path class="g" d="M7 9.5 L2.5 14.5"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+33
View File
@@ -0,0 +1,33 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
- Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
-
- Licensed under the Apache License, Version 2.0 (the "License");
- you may not use this file except in compliance with the License.
- You may obtain a copy of the License at
-
- http://www.apache.org/licenses/LICENSE-2.0
-
- Unless required by applicable law or agreed to in writing, software
- distributed under the License is distributed on an "AS IS" BASIS,
- WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
- See the License for the specific language governing permissions and
- limitations under the License.
-
-->
<svg width="40" height="40" viewBox="0 0 40 40" xmlns="http://www.w3.org/2000/svg" aria-labelledby="title" role="img">
<title>Lyng Plugin Icon (temporary λ)</title>
<defs>
<!-- Monochrome, theme-friendly -->
<style>
.glyph { fill: none; stroke: currentColor; stroke-width: 3; stroke-linecap: round; stroke-linejoin: round; }
</style>
</defs>
<!-- Safe inset to avoid edge clipping in 40x40 canvas -->
<g transform="translate(2,2)">
<!-- Stylized lambda: rising stem + curved tail -->
<path class="glyph" d="M12 6 L18 22 C19.2 25.5 22.2 28 26 28 L32 28"/>
<path class="glyph" d="M18 22 L8 34"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

+88
View File
@@ -0,0 +1,88 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
plugins {
kotlin("jvm")
id("org.jetbrains.intellij") version "1.17.3"
}
group = "net.sergeych.lyng"
version = "0.0.3-SNAPSHOT"
kotlin {
jvmToolchain(17)
}
repositories {
mavenCentral()
// Use the same repositories as the rest of the project so plugin runtime deps resolve
maven("https://maven.universablockchain.com/")
maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
mavenLocal()
}
dependencies {
implementation(project(":lynglib"))
// Include lyngio so Quick Docs can reflectively load fs docs registrar (FsBuiltinDocs)
implementation(project(":lyngio"))
// Rich Markdown renderer for Quick Docs
implementation("com.vladsch.flexmark:flexmark-all:0.64.8")
// Tests for IntelliJ Platform fixtures rely on JUnit 3/4 API (junit.framework.TestCase)
// Add JUnit 4 which contains the JUnit 3 compatibility classes used by BasePlatformTestCase/UsefulTestCase
testImplementation("junit:junit:4.13.2")
}
intellij {
type.set("IC")
// Build against a modern baseline. Install range is controlled by since/until below.
version.set("2024.3.1")
// We manage <idea-version> ourselves in plugin.xml to keep it open-ended (no upper cap)
updateSinceUntilBuild.set(false)
// Include only available bundled plugins for this IDE build
plugins.set(listOf(
"com.intellij.java",
// Provide Grazie API on compile classpath (bundled in 2024.3+, but add here for compilation)
"tanvd.grazi"
// Do not list com.intellij.spellchecker here: it is expected to be bundled with the IDE.
// Listing it causes Gradle to search for a separate plugin artifact and fail on IC 2024.3.
))
}
tasks {
patchPluginXml {
// Keep version and other metadata patched by Gradle, but since/until are controlled in plugin.xml.
// (intellij.updateSinceUntilBuild=false prevents Gradle from injecting an until-build cap)
}
// Build an installable plugin zip and copy it to $PROJECT_ROOT/distributables
// Usage: ./gradlew :lyng-idea:buildInstallablePlugin
// It depends on buildPlugin and overwrites any existing file with the same name
register<Copy>("buildInstallablePlugin") {
dependsOn("buildPlugin")
// The Gradle IntelliJ Plugin produces: build/distributions/<project.name>-<version>.zip
val zipName = "${project.name}-${project.version}.zip"
val sourceZip = layout.buildDirectory.file("distributions/$zipName")
from(sourceZip)
into(rootProject.layout.projectDirectory.dir("distributables"))
// Overwrite if a file with the same name exists
duplicatesStrategy = DuplicatesStrategy.INCLUDE
}
}
@@ -0,0 +1,27 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea
import com.intellij.openapi.fileTypes.LanguageFileType
import javax.swing.Icon
object LyngFileType : LanguageFileType(LyngLanguage) {
override fun getName(): String = "Lyng"
override fun getDescription(): String = "Lyng language file"
override fun getDefaultExtension(): String = "lyng"
override fun getIcon(): Icon? = LyngIcons.FILE
}
@@ -0,0 +1,24 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea
import com.intellij.openapi.util.IconLoader
import javax.swing.Icon
object LyngIcons {
val FILE: Icon = IconLoader.getIcon("/icons/lyng_file.svg", LyngIcons::class.java)
}
@@ -0,0 +1,21 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea
import com.intellij.lang.Language
object LyngLanguage : Language("Lyng")
@@ -0,0 +1,387 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.annotators
import com.intellij.lang.annotation.AnnotationHolder
import com.intellij.lang.annotation.ExternalAnnotator
import com.intellij.lang.annotation.HighlightSeverity
import com.intellij.openapi.editor.Document
import com.intellij.openapi.progress.ProgressManager
import com.intellij.openapi.util.Key
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiFile
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.ScriptError
import net.sergeych.lyng.Source
import net.sergeych.lyng.binding.Binder
import net.sergeych.lyng.binding.SymbolKind
import net.sergeych.lyng.highlight.HighlightKind
import net.sergeych.lyng.highlight.SimpleLyngHighlighter
import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.highlight.LyngHighlighterColors
import net.sergeych.lyng.idea.util.IdeLenientImportProvider
import net.sergeych.lyng.miniast.*
/**
* ExternalAnnotator that runs Lyng MiniAst on the document text in background
* and applies semantic highlighting comparable with the web highlighter.
*/
class LyngExternalAnnotator : ExternalAnnotator<LyngExternalAnnotator.Input, LyngExternalAnnotator.Result>() {
data class Input(val text: String, val modStamp: Long, val previousSpans: List<Span>?)
data class Span(val start: Int, val end: Int, val key: com.intellij.openapi.editor.colors.TextAttributesKey)
data class Error(val start: Int, val end: Int, val message: String)
data class Result(val modStamp: Long, val spans: List<Span>, val error: Error? = null,
val spellIdentifiers: List<IntRange> = emptyList(),
val spellComments: List<IntRange> = emptyList(),
val spellStrings: List<IntRange> = emptyList())
override fun collectInformation(file: PsiFile): Input? {
val doc: Document = file.viewProvider.document ?: return null
val cached = file.getUserData(CACHE_KEY)
// Fast fix (1): reuse cached spans only if they were computed for the same modification stamp
val prev = if (cached != null && cached.modStamp == doc.modificationStamp) cached.spans else null
return Input(doc.text, doc.modificationStamp, prev)
}
override fun doAnnotate(collectedInfo: Input?): Result? {
if (collectedInfo == null) return null
ProgressManager.checkCanceled()
val text = collectedInfo.text
// Build Mini-AST using the same mechanism as web highlighter
val sink = MiniAstBuilder()
val source = Source("<ide>", text)
try {
// Call suspend API from blocking context
val provider = IdeLenientImportProvider.create()
runBlocking { Compiler.compileWithMini(source, provider, sink) }
} catch (e: Throwable) {
if (e is com.intellij.openapi.progress.ProcessCanceledException) throw e
// On script parse error: keep previous spans and report the error location
if (e is ScriptError) {
val off = try { source.offsetOf(e.pos) } catch (_: Throwable) { -1 }
val start0 = off.coerceIn(0, text.length.coerceAtLeast(0))
val (start, end) = expandErrorRange(text, start0)
// Fast fix (5): clear cached highlighting after the error start position
val trimmed = collectedInfo.previousSpans?.filter { it.end <= start } ?: emptyList()
return Result(
collectedInfo.modStamp,
trimmed,
Error(start, end, e.errorMessage)
)
}
// Other failures: keep previous spans without error
return Result(collectedInfo.modStamp, collectedInfo.previousSpans ?: emptyList(), null)
}
ProgressManager.checkCanceled()
val mini = sink.build() ?: return Result(collectedInfo.modStamp, collectedInfo.previousSpans ?: emptyList())
val out = ArrayList<Span>(256)
fun putRange(start: Int, end: Int, key: com.intellij.openapi.editor.colors.TextAttributesKey) {
if (start in 0..end && end <= text.length && start < end) out += Span(start, end, key)
}
fun putName(startPos: net.sergeych.lyng.Pos, name: String, key: com.intellij.openapi.editor.colors.TextAttributesKey) {
val s = source.offsetOf(startPos)
putRange(s, (s + name.length).coerceAtMost(text.length), key)
}
fun putMiniRange(r: MiniRange, key: com.intellij.openapi.editor.colors.TextAttributesKey) {
val s = source.offsetOf(r.start)
val e = source.offsetOf(r.end)
putRange(s, e, key)
}
// Declarations
for (d in mini.declarations) {
when (d) {
is MiniFunDecl -> putName(d.nameStart, d.name, LyngHighlighterColors.FUNCTION_DECLARATION)
is MiniClassDecl -> putName(d.nameStart, d.name, LyngHighlighterColors.TYPE)
is MiniValDecl -> putName(
d.nameStart,
d.name,
if (d.mutable) LyngHighlighterColors.VARIABLE else LyngHighlighterColors.VALUE
)
}
}
// Imports: each segment as namespace/path
for (imp in mini.imports) {
for (seg in imp.segments) putMiniRange(seg.range, LyngHighlighterColors.NAMESPACE)
}
// Parameters
for (fn in mini.declarations.filterIsInstance<MiniFunDecl>()) {
for (p in fn.params) putName(p.nameStart, p.name, LyngHighlighterColors.PARAMETER)
}
// Type name segments (including generics base & args)
fun addTypeSegments(t: MiniTypeRef?) {
when (t) {
is MiniTypeName -> t.segments.forEach { seg ->
val s = source.offsetOf(seg.range.start)
putRange(s, (s + seg.name.length).coerceAtMost(text.length), LyngHighlighterColors.TYPE)
}
is MiniGenericType -> {
addTypeSegments(t.base)
t.args.forEach { addTypeSegments(it) }
}
is MiniFunctionType -> {
t.receiver?.let { addTypeSegments(it) }
t.params.forEach { addTypeSegments(it) }
addTypeSegments(t.returnType)
}
is MiniTypeVar -> { /* name is in range; could be highlighted as TYPE as well */
putMiniRange(t.range, LyngHighlighterColors.TYPE)
}
null -> {}
}
}
for (d in mini.declarations) {
when (d) {
is MiniFunDecl -> {
addTypeSegments(d.returnType)
d.params.forEach { addTypeSegments(it.type) }
}
is MiniValDecl -> addTypeSegments(d.type)
is MiniClassDecl -> {
d.ctorFields.forEach { addTypeSegments(it.type) }
d.classFields.forEach { addTypeSegments(it.type) }
}
}
}
ProgressManager.checkCanceled()
// Semantic usages via Binder (best-effort)
try {
val binding = Binder.bind(text, mini)
// Map declaration ranges to avoid duplicating them as usages
val declKeys = HashSet<Pair<Int, Int>>(binding.symbols.size * 2)
for (sym in binding.symbols) declKeys += (sym.declStart to sym.declEnd)
fun keyForKind(k: SymbolKind) = when (k) {
SymbolKind.Function -> LyngHighlighterColors.FUNCTION
SymbolKind.Class, SymbolKind.Enum -> LyngHighlighterColors.TYPE
SymbolKind.Param -> LyngHighlighterColors.PARAMETER
SymbolKind.Val -> LyngHighlighterColors.VALUE
SymbolKind.Var -> LyngHighlighterColors.VARIABLE
}
// Track covered ranges to not override later heuristics
val covered = HashSet<Pair<Int, Int>>()
for (ref in binding.references) {
val key = ref.start to ref.end
if (declKeys.contains(key)) continue
val sym = binding.symbols.firstOrNull { it.id == ref.symbolId } ?: continue
val color = keyForKind(sym.kind)
putRange(ref.start, ref.end, color)
covered += key
}
// Heuristics on top of binder: function call-sites and simple name-based roles
ProgressManager.checkCanceled()
val tokens = try { SimpleLyngHighlighter().highlight(text) } catch (_: Throwable) { emptyList() }
fun isFollowedByParenOrBlock(rangeEnd: Int): Boolean {
var i = rangeEnd
while (i < text.length) {
val ch = text[i]
if (ch == ' ' || ch == '\t' || ch == '\r' || ch == '\n') { i++; continue }
return ch == '(' || ch == '{'
}
return false
}
// Build simple name -> role map for top-level vals/vars and parameters
val nameRole = HashMap<String, com.intellij.openapi.editor.colors.TextAttributesKey>(8)
for (d in mini.declarations) when (d) {
is MiniValDecl -> nameRole[d.name] = if (d.mutable) LyngHighlighterColors.VARIABLE else LyngHighlighterColors.VALUE
is MiniFunDecl -> d.params.forEach { p -> nameRole[p.name] = LyngHighlighterColors.PARAMETER }
else -> {}
}
for (s in tokens) if (s.kind == HighlightKind.Identifier) {
val start = s.range.start
val end = s.range.endExclusive
val key = start to end
if (key in covered || key in declKeys) continue
// Call-site detection first so it wins over var/param role
if (isFollowedByParenOrBlock(end)) {
putRange(start, end, LyngHighlighterColors.FUNCTION)
covered += key
continue
}
// Simple role by known names
val ident = try { text.substring(start, end) } catch (_: Throwable) { null }
if (ident != null) {
val roleKey = nameRole[ident]
if (roleKey != null) {
putRange(start, end, roleKey)
covered += key
}
}
}
} catch (e: Throwable) {
// Must rethrow cancellation; otherwise ignore binder failures (best-effort)
if (e is com.intellij.openapi.progress.ProcessCanceledException) throw e
}
// Add annotation coloring using token highlighter (treat @Label as annotation)
run {
val tokens = try { SimpleLyngHighlighter().highlight(text) } catch (_: Throwable) { emptyList() }
for (s in tokens) if (s.kind == HighlightKind.Label) {
val start = s.range.start
val end = s.range.endExclusive
if (start in 0..end && end <= text.length && start < end) {
val lexeme = try { text.substring(start, end) } catch (_: Throwable) { null }
if (lexeme != null && lexeme.startsWith("@")) {
putRange(start, end, LyngHighlighterColors.ANNOTATION)
}
}
}
}
// Map Enum constants from token highlighter to IDEA enum constant color
run {
val tokens = try { SimpleLyngHighlighter().highlight(text) } catch (_: Throwable) { emptyList() }
for (s in tokens) if (s.kind == HighlightKind.EnumConstant) {
val start = s.range.start
val end = s.range.endExclusive
if (start in 0..end && end <= text.length && start < end) {
putRange(start, end, LyngHighlighterColors.ENUM_CONSTANT)
}
}
}
// Build spell index payload: identifiers from symbols + references; comments/strings from simple highlighter
val idRanges = mutableSetOf<IntRange>()
try {
val binding = Binder.bind(text, mini)
for (sym in binding.symbols) {
val s = sym.declStart; val e = sym.declEnd
if (s in 0..e && e <= text.length && s < e) idRanges += (s until e)
}
for (ref in binding.references) {
val s = ref.start; val e = ref.end
if (s in 0..e && e <= text.length && s < e) idRanges += (s until e)
}
} catch (_: Throwable) {
// Best-effort; no identifiers if binder fails
}
val tokens = try { SimpleLyngHighlighter().highlight(text) } catch (_: Throwable) { emptyList() }
val commentRanges = tokens.filter { it.kind == HighlightKind.Comment }.map { it.range.start until it.range.endExclusive }
val stringRanges = tokens.filter { it.kind == HighlightKind.String }.map { it.range.start until it.range.endExclusive }
return Result(collectedInfo.modStamp, out, null,
spellIdentifiers = idRanges.toList(),
spellComments = commentRanges,
spellStrings = stringRanges)
}
override fun apply(file: PsiFile, annotationResult: Result?, holder: AnnotationHolder) {
if (annotationResult == null) return
// Skip if cache is up-to-date
val doc = file.viewProvider.document
val currentStamp = doc?.modificationStamp
val cached = file.getUserData(CACHE_KEY)
val result = if (cached != null && currentStamp != null && cached.modStamp == currentStamp) cached else annotationResult
file.putUserData(CACHE_KEY, result)
// Store spell index for spell/grammar engines to consume (suspend until ready)
val ids = result.spellIdentifiers.map { TextRange(it.first, it.last + 1) }
val coms = result.spellComments.map { TextRange(it.first, it.last + 1) }
val strs = result.spellStrings.map { TextRange(it.first, it.last + 1) }
net.sergeych.lyng.idea.spell.LyngSpellIndex.store(file,
net.sergeych.lyng.idea.spell.LyngSpellIndex.Data(
modStamp = result.modStamp,
identifiers = ids,
comments = coms,
strings = strs
)
)
// Optional diagnostic overlay: visualize the ranges we will feed to spellcheckers
val settings = net.sergeych.lyng.idea.settings.LyngFormatterSettings.getInstance(file.project)
if (settings.debugShowSpellFeed) {
fun paint(r: TextRange, label: String) {
holder.newAnnotation(HighlightSeverity.WEAK_WARNING, "spell-feed: $label")
.range(r)
.create()
}
ids.forEach { paint(it, "id") }
coms.forEach { paint(it, "comment") }
if (settings.spellCheckStringLiterals) strs.forEach { paint(it, "string") }
}
for (s in result.spans) {
holder.newSilentAnnotation(HighlightSeverity.INFORMATION)
.range(TextRange(s.start, s.end))
.textAttributes(s.key)
.create()
}
// Show syntax error if present
val err = result.error
if (err != null) {
val start = err.start.coerceIn(0, (doc?.textLength ?: 0))
val end = err.end.coerceIn(start, (doc?.textLength ?: start))
if (end > start) {
holder.newAnnotation(HighlightSeverity.ERROR, err.message)
.range(TextRange(start, end))
.create()
}
}
}
companion object {
private val CACHE_KEY: Key<Result> = Key.create("LYNG_SEMANTIC_CACHE")
}
/**
* Make the error highlight a bit wider than a single character so it is easier to see and click.
* Strategy:
* - If the offset points inside an identifier-like token (letters/digits/underscore), expand to the full token.
* - Otherwise select a small range starting at the offset with a minimum width, but not crossing the line end.
*/
private fun expandErrorRange(text: String, rawStart: Int): Pair<Int, Int> {
if (text.isEmpty()) return 0 to 0
val len = text.length
val start = rawStart.coerceIn(0, len)
fun isWord(ch: Char) = ch == '_' || ch.isLetterOrDigit()
if (start < len && isWord(text[start])) {
var s = start
var e = start
while (s > 0 && isWord(text[s - 1])) s--
while (e < len && isWord(text[e])) e++
return s to e
}
// Not inside a word: select a short, visible range up to EOL
val lineEnd = text.indexOf('\n', start).let { if (it == -1) len else it }
val minWidth = 4
val end = (start + minWidth).coerceAtMost(lineEnd).coerceAtLeast((start + 1).coerceAtMost(lineEnd))
return start to end
}
}
@@ -0,0 +1,27 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.comment
import com.intellij.lang.Commenter
class LyngCommenter : Commenter {
override fun getLineCommentPrefix(): String = "//"
override fun getBlockCommentPrefix(): String = "/*"
override fun getBlockCommentSuffix(): String = "*/"
override fun getCommentedBlockCommentPrefix(): String? = null
override fun getCommentedBlockCommentSuffix(): String? = null
}
@@ -0,0 +1,934 @@
/*
* Lightweight BASIC completion for Lyng, MVP version.
* Uses MiniAst (best-effort) + BuiltinDocRegistry to suggest symbols.
*/
package net.sergeych.lyng.idea.completion
import com.intellij.codeInsight.completion.*
import com.intellij.codeInsight.lookup.LookupElementBuilder
import com.intellij.icons.AllIcons
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Document
import com.intellij.openapi.util.Key
import com.intellij.patterns.PlatformPatterns
import com.intellij.psi.PsiFile
import com.intellij.util.ProcessingContext
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Source
import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import net.sergeych.lyng.idea.util.DocsBootstrap
import net.sergeych.lyng.idea.util.IdeLenientImportProvider
import net.sergeych.lyng.idea.util.TextCtx
import net.sergeych.lyng.miniast.*
class LyngCompletionContributor : CompletionContributor() {
init {
extend(
CompletionType.BASIC,
PlatformPatterns.psiElement().withLanguage(LyngLanguage),
Provider
)
}
private object Provider : CompletionProvider<CompletionParameters>() {
private val log = Logger.getInstance(LyngCompletionContributor::class.java)
private const val DEBUG_COMPLETION = false
override fun addCompletions(
parameters: CompletionParameters,
context: ProcessingContext,
result: CompletionResultSet
) {
// Ensure external/bundled docs are registered (e.g., lyng.io.fs with Path)
DocsBootstrap.ensure()
// Ensure stdlib Obj*-defined docs (e.g., String methods via ObjString.addFnDoc) are initialized
StdlibDocsBootstrap.ensure()
val file: PsiFile = parameters.originalFile
if (file.language != LyngLanguage) return
// Feature toggle: allow turning completion off from settings
val settings = LyngFormatterSettings.getInstance(file.project)
if (!settings.enableLyngCompletionExperimental) return
val document: Document = file.viewProvider.document ?: return
val text = document.text
val caret = parameters.offset.coerceIn(0, text.length)
val prefix = TextCtx.prefixAt(text, caret)
val withPrefix = result.withPrefixMatcher(prefix).caseInsensitive()
// Emission with cap
val cap = 200
var added = 0
val emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit = { le ->
if (added < cap) {
withPrefix.addElement(le)
added++
}
}
// Determine if we are in member context (dot before caret or before word start)
val wordRange = TextCtx.wordRangeAt(text, caret)
val memberDotPos = (wordRange?.let { TextCtx.findDotLeft(text, it.startOffset) })
?: TextCtx.findDotLeft(text, caret)
if (DEBUG_COMPLETION) {
log.info("[LYNG_DEBUG] Completion: caret=$caret prefix='${prefix}' memberDotPos=${memberDotPos} file='${file.name}'")
}
// Build MiniAst (cached) for both global and member contexts to enable local class/val inference
val mini = buildMiniAstCached(file, text)
// Delegate computation to the shared engine to keep behavior in sync with tests
val engineItems = try {
runBlocking { CompletionEngineLight.completeSuspend(text, caret) }
} catch (t: Throwable) {
if (DEBUG_COMPLETION) log.warn("[LYNG_DEBUG] Engine completion failed: ${t.message}")
emptyList()
}
if (DEBUG_COMPLETION) {
val preview = engineItems.take(10).joinToString { it.name }
log.info("[LYNG_DEBUG] Engine items: count=${engineItems.size} preview=[${preview}]")
}
// If we are in member context and the engine produced nothing, try a guarded local fallback
if (memberDotPos != null && engineItems.isEmpty()) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback: engine returned 0 in member context; trying local inference")
// Build imported modules from text (lenient) + stdlib; avoid heavy MiniAst here
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
// Try inferring return/receiver class around the dot
val inferred =
// Prefer MiniAst-based inference (return type from member call or receiver type)
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?:
guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (inferred != null) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback inferred receiver/return class='$inferred' — offering its members")
offerMembers(emit, imported, inferred, sourceText = text, mini = mini)
return
} else {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Fallback could not infer class; keeping list empty (no globals after dot)")
return
}
}
// In global context, add params in scope first (engine does not include them)
if (memberDotPos == null && mini != null) {
offerParamsInScope(emit, mini, text, caret)
}
// Render engine items
for (ci in engineItems) {
val builder = when (ci.kind) {
Kind.Function -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Function)
.let { b -> if (!ci.tailText.isNullOrBlank()) b.withTailText(ci.tailText, true) else b }
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
.withInsertHandler(ParenInsertHandler)
Kind.Method -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Method)
.let { b -> if (!ci.tailText.isNullOrBlank()) b.withTailText(ci.tailText, true) else b }
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
.withInsertHandler(ParenInsertHandler)
Kind.Class_ -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Class)
Kind.Value -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Field)
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
Kind.Field -> LookupElementBuilder.create(ci.name)
.withIcon(AllIcons.Nodes.Field)
.let { b -> if (!ci.typeText.isNullOrBlank()) b.withTypeText(ci.typeText, true) else b }
}
emit(builder)
}
// In member context, ensure stdlib extension-like methods (e.g., String.re) are present
if (memberDotPos != null) {
val existing = engineItems.map { it.name }.toMutableSet()
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
val inferredClass =
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?: guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (!inferredClass.isNullOrBlank()) {
val ext = BuiltinDocRegistry.extensionMethodNamesFor(inferredClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Post-engine extension check for $inferredClass: ${'$'}{ext}")
for (name in ext) {
if (existing.contains(name)) continue
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, inferredClass, name)
if (resolved != null) {
when (val member = resolved.second) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${ '$' }params)", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
existing.add(name)
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (member.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
existing.add(name)
}
}
} else {
// Fallback: emit simple method name without detailed types
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
existing.add(name)
}
}
}
}
// If in member context and engine items are suspiciously sparse, try to enrich via local inference + offerMembers
if (memberDotPos != null && engineItems.size < 3) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Engine produced only ${engineItems.size} items in member context — trying enrichment")
val fromText = extractImportsFromText(text)
val imported = LinkedHashSet<String>().apply {
fromText.forEach { add(it) }
add("lyng.stdlib")
}.toList()
val inferred =
guessReturnClassFromMemberCallBeforeMini(mini, text, memberDotPos, imported)
?: guessReceiverClassViaMini(mini, text, memberDotPos, imported)
?: guessReturnClassFromMemberCallBefore(text, memberDotPos, imported)
?: guessReturnClassFromTopLevelCallBefore(text, memberDotPos, imported)
?: guessReturnClassAcrossKnownCallees(text, memberDotPos, imported)
?: guessReceiverClass(text, memberDotPos, imported)
if (inferred != null) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Enrichment inferred class='$inferred' — offering its members")
offerMembers(emit, imported, inferred, sourceText = text, mini = mini)
}
}
return
}
private fun offerDecl(emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit, d: MiniDecl) {
val name = d.name
val builder = when (d) {
is MiniFunDecl -> {
val params = d.params.joinToString(", ") { it.name }
val ret = typeOf(d.returnType)
val tail = "(${params})"
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Function)
.withTailText(tail, true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
}
is MiniClassDecl -> LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Class)
is MiniValDecl -> {
val kindIcon = if (d.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field
LookupElementBuilder.create(name)
.withIcon(kindIcon)
.withTypeText(typeOf(d.type), true)
}
else -> LookupElementBuilder.create(name)
}
emit(builder)
}
private object ParenInsertHandler : InsertHandler<com.intellij.codeInsight.lookup.LookupElement> {
override fun handleInsert(context: InsertionContext, item: com.intellij.codeInsight.lookup.LookupElement) {
val doc = context.document
val tailOffset = context.tailOffset
val nextChar = doc.charsSequence.getOrNull(tailOffset)
if (nextChar != '(') {
doc.insertString(tailOffset, "()")
context.editor.caretModel.moveToOffset(tailOffset + 1)
}
}
}
// --- Member completion helpers ---
private fun offerMembers(
emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit,
imported: List<String>,
className: String,
staticOnly: Boolean = false
, sourceText: String,
mini: MiniScript? = null
) {
// Ensure modules are seeded in the registry (triggers lazy stdlib build too)
for (m in imported) BuiltinDocRegistry.docsForModule(m)
val classes = DocLookupUtils.aggregateClasses(imported)
if (DEBUG_COMPLETION) {
val keys = classes.keys.joinToString(", ")
log.info("[LYNG_DEBUG] offerMembers: imported=${imported} classes=[${keys}] target=${className}")
}
val visited = mutableSetOf<String>()
// Collect separated to keep tiers: direct first, then inherited
val directMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
val inheritedMap = LinkedHashMap<String, MutableList<MiniMemberDecl>>()
// 0) Prefer locally-declared class members (same-file) when available
val localClass = mini?.declarations?.filterIsInstance<MiniClassDecl>()?.firstOrNull { it.name == className }
if (localClass != null) {
for (m in localClass.members) {
val list = directMap.getOrPut(m.name) { mutableListOf() }
list.add(m)
}
// If MiniAst didn't populate members (empty), try to scan class body text for member signatures
if (localClass.members.isEmpty()) {
val scanned = scanLocalClassMembersFromText(mini, text = sourceText, cls = localClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Local scan for class ${localClass.name}: found ${scanned.size} members -> ${scanned.keys}")
for ((name, sig) in scanned) {
when (sig.kind) {
"fun" -> {
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(" + (sig.params?.joinToString(", ") ?: "") + ")", true)
.let { b -> sig.typeText?.let { b.withTypeText(": $it", true) } ?: b }
.withInsertHandler(ParenInsertHandler)
emit(builder)
}
"val", "var" -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (sig.kind == "var") AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.let { b -> sig.typeText?.let { b.withTypeText(": $it", true) } ?: b }
emit(builder)
}
}
}
}
}
fun addMembersOf(clsName: String, tierDirect: Boolean) {
val cls = classes[clsName] ?: return
val target = if (tierDirect) directMap else inheritedMap
for (m in cls.members) {
if (staticOnly) {
// Filter only static members in namespace/static context
when (m) {
is MiniMemberFunDecl -> if (!m.isStatic) continue
is MiniMemberValDecl -> if (!m.isStatic) continue
}
}
val list = target.getOrPut(m.name) { mutableListOf() }
list.add(m)
}
// Then inherited
for (base in cls.bases) {
if (visited.add(base)) addMembersOf(base, false)
}
}
visited.add(className)
addMembersOf(className, true)
if (DEBUG_COMPLETION) {
log.info("[LYNG_DEBUG] offerMembers: direct=${directMap.size} inherited=${inheritedMap.size} for ${className}")
}
// If the docs model lacks explicit bases for some core container classes,
// conservatively supplement with preferred parents to expose common ops.
fun supplementPreferredBases(receiver: String) {
// Preference/known lineage map kept tiny and safe
val extras = when (receiver) {
"List" -> listOf("Collection", "Iterable")
"Array" -> listOf("Collection", "Iterable")
// In practice, many high-level ops users expect on iteration live on Iterable.
// For editor assistance, expose Iterable ops for Iterator receivers too.
"Iterator" -> listOf("Iterable")
else -> emptyList()
}
for (base in extras) {
if (visited.add(base)) addMembersOf(base, false)
}
}
supplementPreferredBases(className)
fun emitGroup(map: LinkedHashMap<String, MutableList<MiniMemberDecl>>) {
val keys = map.keys.sortedBy { it.lowercase() }
for (name in keys) {
val list = map[name] ?: continue
// Choose a representative for display:
// 1) Prefer a method with a known return type
// 2) Else any method
// 3) Else the first variant
val rep =
list.asSequence()
.filterIsInstance<MiniMemberFunDecl>()
.firstOrNull { it.returnType != null }
?: list.firstOrNull { it is MiniMemberFunDecl }
?: list.first()
when (rep) {
is MiniMemberFunDecl -> {
val params = rep.params.joinToString(", ") { it.name }
val ret = typeOf(rep.returnType)
val extra = list.count { it is MiniMemberFunDecl } - 1
val overloads = if (extra > 0) " (+$extra overloads)" else ""
val tail = "(${params})$overloads"
val icon = AllIcons.Nodes.Method
val builder = LookupElementBuilder.create(name)
.withIcon(icon)
.withTailText(tail, true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
}
is MiniMemberValDecl -> {
val icon = if (rep.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field
// Prefer a field variant with known type if available
val chosen = list.asSequence()
.filterIsInstance<MiniMemberValDecl>()
.firstOrNull { it.type != null } ?: rep
val builder = LookupElementBuilder.create(name)
.withIcon(icon)
.withTypeText(typeOf((chosen as MiniMemberValDecl).type), true)
emit(builder)
}
}
}
}
// Emit what we have first
emitGroup(directMap)
emitGroup(inheritedMap)
// If suggestions are suspiciously sparse for known container classes,
// try to conservatively supplement using a curated list resolved via docs registry.
val totalSuggested = directMap.size + inheritedMap.size
val isContainer = className in setOf("Iterator", "Iterable", "Collection", "List", "Array")
if (isContainer && totalSuggested < 3) {
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Supplementing members for $className; had=$totalSuggested")
val common = when (className) {
"Iterator" -> listOf(
"hasNext", "next", "forEach", "map", "filter", "take", "drop", "toList", "count", "any", "all"
)
else -> listOf(
// Iterable/Collection/List/Array common ops
"size", "isEmpty", "map", "flatMap", "filter", "first", "last", "contains",
"any", "all", "count", "forEach", "toList", "toSet"
)
}
val already = (directMap.keys + inheritedMap.keys).toMutableSet()
for (name in common) {
if (name in already) continue
// Try resolve across classes first to get types/params; if it fails, emit a synthetic safe suggestion.
val resolved = DocLookupUtils.findMemberAcrossClasses(imported, name)
if (resolved != null) {
val member = resolved.second
when (member) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${params})", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
already.add(name)
}
}
} else {
// Synthetic fallback: method without detailed params/types to improve UX in absence of docs
val isProperty = name in setOf("size", "length")
val builder = if (isProperty) {
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Field)
} else {
LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
}
emit(builder)
already.add(name)
}
}
}
// Supplement with stdlib extension-like methods defined in root.lyng (e.g., fun String.trim(...))
run {
val already = (directMap.keys + inheritedMap.keys).toMutableSet()
val ext = BuiltinDocRegistry.extensionMethodNamesFor(className)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Extensions for $className: count=${ext.size} -> ${ext}")
for (name in ext) {
if (already.contains(name)) continue
// Try to resolve full signature via registry first to get params and return type
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, className, name)
if (resolved != null) {
when (val member = resolved.second) {
is MiniMemberFunDecl -> {
val params = member.params.joinToString(", ") { it.name }
val ret = typeOf(member.returnType)
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("(${params})", true)
.withTypeText(ret, true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
continue
}
is MiniMemberValDecl -> {
val builder = LookupElementBuilder.create(name)
.withIcon(if (member.mutable) AllIcons.Nodes.Variable else AllIcons.Nodes.Field)
.withTypeText(typeOf(member.type), true)
emit(builder)
already.add(name)
continue
}
}
}
// Fallback: emit without detailed types if we couldn't resolve
val builder = LookupElementBuilder.create(name)
.withIcon(AllIcons.Nodes.Method)
.withTailText("()", true)
.withInsertHandler(ParenInsertHandler)
emit(builder)
already.add(name)
}
}
}
// --- MiniAst-based inference helpers ---
private fun previousIdentifierBeforeDot(text: String, dotPos: Int): String? {
var i = dotPos - 1
// skip whitespace
while (i >= 0 && text[i].isWhitespace()) i--
val end = i + 1
while (i >= 0 && TextCtx.isIdentChar(text[i])) i--
val start = i + 1
return if (start < end) text.substring(start, end) else null
}
private fun guessReceiverClassViaMini(mini: MiniScript?, text: String, dotPos: Int, imported: List<String>): String? {
if (mini == null) return null
val ident = previousIdentifierBeforeDot(text, dotPos) ?: return null
// 1) Local val/var in the file
val valDecl = mini.declarations.filterIsInstance<MiniValDecl>().firstOrNull { it.name == ident }
val typeFromVal = valDecl?.type?.let { simpleClassNameOf(it) }
if (!typeFromVal.isNullOrBlank()) return typeFromVal
// If initializer exists, try to sniff ClassName(
val initR = valDecl?.initRange
if (initR != null) {
val src = mini.range.start.source
val s = src.offsetOf(initR.start)
val e = src.offsetOf(initR.end).coerceAtMost(text.length)
if (s in 0..e && e <= text.length) {
val init = text.substring(s, e)
Regex("([A-Za-z_][A-Za-z0-9_]*)\\s*\\(").find(init)?.let { m ->
val cls = m.groupValues[1]
return cls
}
}
}
// 2) Parameters in any function (best-effort without scope mapping)
val paramType = mini.declarations.filterIsInstance<MiniFunDecl>()
.asSequence()
.flatMap { it.params.asSequence() }
.firstOrNull { it.name == ident }?.type
val typeFromParam = simpleClassNameOf(paramType)
if (!typeFromParam.isNullOrBlank()) return typeFromParam
return null
}
private fun guessReturnClassFromMemberCallBeforeMini(mini: MiniScript?, text: String, dotPos: Int, imported: List<String>): String? {
if (mini == null) return null
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// back to matching '('
i--
var depth = 0
while (i >= 0) {
when (text[i]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Ensure member call: dot before callee
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k < 0 || text[k] != '.') return null
val prevDot = k
// Resolve receiver class via MiniAst (ident like `x`)
val receiverClass = guessReceiverClassViaMini(mini, text, prevDot, imported) ?: return null
// If receiver class is a locally declared class, resolve member on it
val localClass = mini.declarations.filterIsInstance<MiniClassDecl>().firstOrNull { it.name == receiverClass }
if (localClass != null) {
val mm = localClass.members.firstOrNull { it.name == callee }
if (mm != null) {
val rt = when (mm) {
is MiniMemberFunDecl -> mm.returnType
is MiniMemberValDecl -> mm.type
else -> null
}
return simpleClassNameOf(rt)
} else {
// Try to scan class body text for method signature and extract return type
val sigs = scanLocalClassMembersFromText(mini, text, localClass)
if (DEBUG_COMPLETION) log.info("[LYNG_DEBUG] Local scan for return type in ${receiverClass}.${callee}: candidates=${sigs.keys}")
val sig = sigs[callee]
if (sig != null && sig.typeText != null) return sig.typeText
}
}
// Else fallback to registry-based resolution (covers imported classes)
return DocLookupUtils.resolveMemberWithInheritance(imported, receiverClass, callee)?.second?.let { m ->
val rt = when (m) {
is MiniMemberFunDecl -> m.returnType
is MiniMemberValDecl -> m.type
}
simpleClassNameOf(rt)
}
}
private data class ScannedSig(val kind: String, val params: List<String>?, val typeText: String?)
private fun scanLocalClassMembersFromText(mini: MiniScript, text: String, cls: MiniClassDecl): Map<String, ScannedSig> {
val src = mini.range.start.source
val start = src.offsetOf(cls.bodyRange?.start ?: cls.range.start)
val end = src.offsetOf(cls.bodyRange?.end ?: cls.range.end).coerceAtMost(text.length)
if (start !in 0..end) return emptyMap()
val body = text.substring(start, end)
val map = LinkedHashMap<String, ScannedSig>()
// fun name(params): Type
val funRe = Regex("(?m)^\\s*fun\\s+([A-Za-z_][A-Za-z0-9_]*)\\s*\\(([^)]*)\\)\\s*(?::\\s*([A-Za-z_][A-Za-z0-9_]*))?")
for (m in funRe.findAll(body)) {
val name = m.groupValues.getOrNull(1) ?: continue
val params = m.groupValues.getOrNull(2)?.split(',')?.mapNotNull { it.trim().takeIf { it.isNotEmpty() } } ?: emptyList()
val type = m.groupValues.getOrNull(3)?.takeIf { it.isNotBlank() }
map[name] = ScannedSig("fun", params, type)
}
// val/var name: Type
val valRe = Regex("(?m)^\\s*(val|var)\\s+([A-Za-z_][A-Za-z0-9_]*)\\s*(?::\\s*([A-Za-z_][A-Za-z0-9_]*))?")
for (m in valRe.findAll(body)) {
val kind = m.groupValues.getOrNull(1) ?: continue
val name = m.groupValues.getOrNull(2) ?: continue
val type = m.groupValues.getOrNull(3)?.takeIf { it.isNotBlank() }
map.putIfAbsent(name, ScannedSig(kind, null, type))
}
return map
}
private fun guessReceiverClass(text: String, dotPos: Int, imported: List<String>): String? {
// 1) Try call-based: ClassName(...).
DocLookupUtils.guessClassFromCallBefore(text, dotPos, imported)?.let { return it }
// 2) Literal heuristics based on the immediate char before '.'
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i >= 0) {
when (text[i]) {
'"' -> {
// Either regular or triple-quoted string; both map to String
return "String"
}
']' -> return "List" // very rough heuristic
'}' -> return "Dict" // map/dictionary literal heuristic
')' -> {
// Parenthesized expression: walk back to matching '(' and inspect inner expression
var j = i - 1
var depth = 0
while (j >= 0) {
when (text[j]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
j--
}
if (j >= 0 && text[j] == '(') {
val innerS = (j + 1).coerceAtLeast(0)
val innerE = i.coerceAtMost(text.length)
if (innerS < innerE) {
val inner = text.substring(innerS, innerE).trim()
if (inner.startsWith('"') && inner.endsWith('"')) return "String"
if (inner.startsWith('[') && inner.endsWith(']')) return "List"
if (inner.startsWith('{') && inner.endsWith('}')) return "Dict"
}
}
}
}
// Numeric literal: support decimal, hex (0x..), and scientific notation (1e-3)
var j = i
var hasDigits = false
var hasDot = false
var hasExp = false
// Walk over digits, letters for hex, dots, and exponent markers
while (j >= 0) {
val ch = text[j]
if (ch.isDigit()) { hasDigits = true; j-- ; continue }
if (ch == '.') { hasDot = true; j-- ; continue }
if (ch == 'e' || ch == 'E') { hasExp = true; j-- ; // optional sign directly before digits
if (j >= 0 && (text[j] == '+' || text[j] == '-')) j--
continue
}
if (ch in listOf('x','X')) { // part of 0x prefix
j--
continue
}
if (ch == 'a' || ch == 'b' || ch == 'c' || ch == 'd' || ch == 'f' ||
ch == 'A' || ch == 'B' || ch == 'C' || ch == 'D' || ch == 'F') {
// hex digit in 0x...
j--
continue
}
break
}
// Now check for 0x/0X prefix
val k = j
val isHex = k >= 1 && text[k] == '0' && (text[k+1] == 'x' || text[k+1] == 'X')
if (hasDigits) {
return if (isHex) "Int" else if (hasDot || hasExp) "Real" else "Int"
}
}
return null
}
/**
* Try to infer the class of the return value of the member call immediately before the dot.
* Example: `Path(".." ).lines().<caret>` → detects `lines()` on receiver class `Path` and returns `Iterator`.
*/
private fun guessReturnClassFromMemberCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0) return null
// We expect a call just before the dot, i.e., ')' ... '.'
if (text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Identify callee identifier just before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Ensure it's a member call (there must be a dot immediately before the callee, ignoring spaces)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k < 0 || text[k] != '.') return null
val prevDot = k
// Infer receiver class at the previous dot
val receiverClass = guessReceiverClass(text, prevDot, imported) ?: return null
// Resolve the callee as a member of receiver class, including inheritance
val resolved = DocLookupUtils.resolveMemberWithInheritance(imported, receiverClass, callee) ?: return null
val member = resolved.second
val returnType = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(returnType)
}
/**
* Infer return class of a top-level call right before the dot: e.g., `files().<caret>`.
* We extract callee name and resolve it among imported modules' top-level functions.
*/
private fun guessReturnClassFromTopLevelCallBefore(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Extract callee ident before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// If it's a member call, bail out (handled in member-call inference)
var k = start - 1
while (k >= 0 && text[k].isWhitespace()) k--
if (k >= 0 && text[k] == '.') return null
// Resolve top-level function in imported modules
for (mod in imported) {
val decls = BuiltinDocRegistry.docsForModule(mod)
val fn = decls.asSequence().filterIsInstance<MiniFunDecl>().firstOrNull { it.name == callee }
if (fn != null) return simpleClassNameOf(fn.returnType)
}
return null
}
/**
* Fallback: if we can at least extract a callee name before the dot and it exists across common classes,
* derive its return type using cross-class lookup (Iterable/Iterator/List preference). This ignores the receiver.
* Example: `something.lines().<caret>` where `something` type is unknown, but `lines()` commonly returns Iterator<String>.
*/
private fun guessReturnClassAcrossKnownCallees(text: String, dotPos: Int, imported: List<String>): String? {
var i = TextCtx.prevNonWs(text, dotPos - 1)
if (i < 0 || text[i] != ')') return null
// Walk back to matching '('
i--
var depth = 0
while (i >= 0) {
val ch = text[i]
when (ch) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
i--
}
if (i < 0 || text[i] != '(') return null
// Extract callee ident before '('
var j = i - 1
while (j >= 0 && text[j].isWhitespace()) j--
val end = j + 1
while (j >= 0 && TextCtx.isIdentChar(text[j])) j--
val start = j + 1
if (start >= end) return null
val callee = text.substring(start, end)
// Try cross-class resolution
val resolved = DocLookupUtils.findMemberAcrossClasses(imported, callee) ?: return null
val member = resolved.second
val returnType = when (member) {
is MiniMemberFunDecl -> member.returnType
is MiniMemberValDecl -> member.type
}
return simpleClassNameOf(returnType)
}
/** Convert a MiniTypeRef to a simple class name as used by docs (e.g., Iterator from Iterator<String>). */
private fun simpleClassNameOf(t: MiniTypeRef?): String? = when (t) {
null -> null
is MiniTypeName -> t.segments.lastOrNull()?.name
is MiniGenericType -> simpleClassNameOf(t.base)
is MiniFunctionType -> null
is MiniTypeVar -> null
}
private fun buildMiniAst(text: String): MiniScript? {
return try {
val sink = MiniAstBuilder()
val provider = IdeLenientImportProvider.create()
val src = Source("<ide>", text)
runBlocking { Compiler.compileWithMini(src, provider, sink) }
sink.build()
} catch (_: Throwable) {
null
}
}
// Cached per PsiFile by document modification stamp
private val MINI_KEY = Key.create<MiniScript>("lyng.mini.cache")
private val STAMP_KEY = Key.create<Long>("lyng.mini.cache.stamp")
private fun buildMiniAstCached(file: PsiFile, text: String): MiniScript? {
val doc = file.viewProvider.document ?: return null
val stamp = doc.modificationStamp
val prevStamp = file.getUserData(STAMP_KEY)
val cached = file.getUserData(MINI_KEY)
if (cached != null && prevStamp != null && prevStamp == stamp) return cached
val built = buildMiniAst(text)
// Cache even null? avoid caching failures; only cache non-null
if (built != null) {
file.putUserData(MINI_KEY, built)
file.putUserData(STAMP_KEY, stamp)
}
return built
}
private fun offerParamsInScope(emit: (com.intellij.codeInsight.lookup.LookupElement) -> Unit, mini: MiniScript, text: String, caret: Int) {
val src = mini.range.start.source
// Find function whose body contains caret or whose whole range contains caret
val fns = mini.declarations.filterIsInstance<MiniFunDecl>()
for (fn in fns) {
val start = src.offsetOf(fn.range.start)
val end = src.offsetOf(fn.range.end).coerceAtMost(text.length)
if (caret in start..end) {
for (p in fn.params) {
val builder = LookupElementBuilder.create(p.name)
.withIcon(AllIcons.Nodes.Variable)
.withTypeText(typeOf(p.type), true)
emit(builder)
}
return
}
}
}
// Lenient textual import extractor (duplicated from QuickDoc privately)
private fun extractImportsFromText(text: String): List<String> {
val result = LinkedHashSet<String>()
val re = Regex("(?m)^\\s*import\\s+([a-zA-Z_][a-zA-Z0-9_]*(?:\\.[a-zA-Z_][a-zA-Z0-9_]*)*)")
re.findAll(text).forEach { m ->
val raw = m.groupValues.getOrNull(1)?.trim().orEmpty()
if (raw.isNotEmpty()) {
val canon = if (raw.startsWith("lyng.")) raw else "lyng.$raw"
result.add(canon)
}
}
return result.toList()
}
private fun typeOf(t: MiniTypeRef?): String {
return when (t) {
null -> ""
is MiniTypeName -> t.segments.lastOrNull()?.name?.let { ": $it" } ?: ""
is MiniGenericType -> {
val base = typeOf(t.base).removePrefix(": ")
val args = t.args.joinToString(",") { typeOf(it).removePrefix(": ") }
": ${base}<${args}>"
}
is MiniFunctionType -> ": (fn)"
is MiniTypeVar -> ": ${t.name}"
}
}
}
}
@@ -0,0 +1,55 @@
/*
* Minimal fallback docs seeding for `lyng.io.fs` used only inside the IDEA plugin
* when external docs module (lyngio) is not present on the classpath.
*
* We keep it tiny and plugin-local to avoid coupling core library to external packages.
*/
package net.sergeych.lyng.idea.docs
import net.sergeych.lyng.miniast.BuiltinDocRegistry
import net.sergeych.lyng.miniast.ParamDoc
import net.sergeych.lyng.miniast.TypeGenericDoc
import net.sergeych.lyng.miniast.type
internal object FsDocsFallback {
@Volatile
private var seeded = false
fun ensureOnce(): Boolean {
if (seeded) return true
synchronized(this) {
if (seeded) return true
BuiltinDocRegistry.module("lyng.io.fs") {
// Class Path summary and a few commonly used methods
classDoc(name = "Path", doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`.") {
method(name = "exists", doc = "Whether the path exists on the filesystem.", returns = type("lyng.Bool"))
method(name = "isFile", doc = "Whether the path exists and is a file.", returns = type("lyng.Bool"))
method(name = "isDir", doc = "Whether the path exists and is a directory.", returns = type("lyng.Bool"))
method(name = "readUtf8", doc = "Read the entire file as UTF-8 string.", returns = type("lyng.String"))
method(
name = "writeUtf8",
doc = "Write UTF-8 string to the file (overwrite).",
params = listOf(ParamDoc("text", type("lyng.String")))
)
method(
name = "bytes",
doc = "Iterate file content as `Buffer` chunks.",
params = listOf(ParamDoc("size", type("lyng.Int"))),
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.Buffer")))
)
method(
name = "lines",
doc = "Iterate file as lines of text.",
returns = TypeGenericDoc(type("lyng.Iterator"), listOf(type("lyng.String")))
)
}
// Top-level exported constants
valDoc(name = "Path", doc = "Filesystem path class. Construct with a string: `Path(\"/tmp\")`.", type = type("Path"))
valDoc(name = "Paths", doc = "Alias of `Path` for those who prefer plural form.", type = type("Path"))
}
seeded = true
return true
}
}
}
@@ -0,0 +1,564 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.docs
import com.intellij.lang.documentation.AbstractDocumentationProvider
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import kotlinx.coroutines.runBlocking
import net.sergeych.lyng.Compiler
import net.sergeych.lyng.Pos
import net.sergeych.lyng.Source
import net.sergeych.lyng.highlight.offsetOf
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.util.IdeLenientImportProvider
import net.sergeych.lyng.idea.util.TextCtx
import net.sergeych.lyng.miniast.*
/**
* Quick Docs backed by MiniAst: when caret is on an identifier that corresponds
* to a declaration name or parameter, render a simple HTML with kind, signature,
* and doc summary if present.
*/
class LyngDocumentationProvider : AbstractDocumentationProvider() {
private val log = Logger.getInstance(LyngDocumentationProvider::class.java)
// Toggle to trace inheritance-based resolutions in Quick Docs. Keep false for normal use.
private val DEBUG_INHERITANCE = false
// Global Quick Doc debug toggle (OFF by default). When false, [LYNG_DEBUG] logs are suppressed.
private val DEBUG_LOG = false
override fun generateDoc(element: PsiElement?, originalElement: PsiElement?): String? {
// Try load external docs registrars (e.g., lyngio) if present on classpath
ensureExternalDocsRegistered()
// Ensure stdlib Obj*-defined docs (e.g., String methods via ObjString.addFnDoc) are initialized
try {
net.sergeych.lyng.miniast.StdlibDocsBootstrap.ensure()
} catch (_: Throwable) {
// best-effort; absence must not break Quick Doc
}
if (element == null) return null
val file: PsiFile = element.containingFile ?: return null
val document: Document = file.viewProvider.document ?: return null
val text = document.text
// Determine caret/lookup offset from the element range
val offset = originalElement?.textRange?.startOffset ?: element.textRange.startOffset
val idRange = TextCtx.wordRangeAt(text, offset) ?: run {
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: no word at offset=$offset in ${file.name}")
return null
}
if (idRange.isEmpty) return null
val ident = text.substring(idRange.startOffset, idRange.endOffset)
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: ident='$ident' at ${idRange.startOffset}..${idRange.endOffset} in ${file.name}")
// Build MiniAst for this file (fast and resilient). Best-effort; on failure continue with registry lookup only.
val sink = MiniAstBuilder()
// Use lenient import provider so unresolved imports (e.g., lyng.io.fs) don't break docs
val provider = IdeLenientImportProvider.create()
val src = Source("<ide>", text)
var mini: MiniScript? = try {
runBlocking { Compiler.compileWithMini(src, provider, sink) }
sink.build()
} catch (t: Throwable) {
// Do not bail out completely: we still can resolve built-in and imported docs (e.g., println)
if (DEBUG_LOG) log.warn("[LYNG_DEBUG] QuickDoc: compileWithMini failed: ${t.message}")
null
}
val haveMini = mini != null
if (mini == null) {
// Ensure we have a dummy script object to avoid NPE in downstream helpers that expect a MiniScript
mini = MiniScript(MiniRange(Pos(src, 1, 1), Pos(src, 1, 1)))
}
val source = src
// Try resolve to: function param at position, function/class/val declaration at position
// 1) Check declarations whose name range contains offset
if (haveMini) for (d in mini.declarations) {
val s = source.offsetOf(d.nameStart)
val e = (s + d.name.length).coerceAtMost(text.length)
if (offset in s until e) {
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: matched decl '${d.name}' kind=${d::class.simpleName}")
return renderDeclDoc(d)
}
}
// 2) Check parameters of functions
if (haveMini) for (fn in mini.declarations.filterIsInstance<MiniFunDecl>()) {
for (p in fn.params) {
val s = source.offsetOf(p.nameStart)
val e = (s + p.name.length).coerceAtMost(text.length)
if (offset in s until e) {
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: matched param '${p.name}' in fun '${fn.name}'")
return renderParamDoc(fn, p)
}
}
}
// 3) Member-context resolution first (dot immediately before identifier): handle literals and calls
run {
val dotPos = TextCtx.findDotLeft(text, idRange.startOffset)
?: TextCtx.findDotLeft(text, offset)
if (dotPos != null) {
// Build imported modules (MiniAst-derived if available, else lenient from text) and ensure stdlib is present
var importedModules = if (haveMini) DocLookupUtils.canonicalImportedModules(mini) else emptyList()
if (importedModules.isEmpty()) {
val fromText = extractImportsFromText(text)
importedModules = if (fromText.isEmpty()) listOf("lyng.stdlib") else fromText
}
if (!importedModules.contains("lyng.stdlib")) importedModules = importedModules + "lyng.stdlib"
// Try literal and call-based receiver inference around the dot
val i = TextCtx.prevNonWs(text, dotPos - 1)
val className: String? = when {
i >= 0 && text[i] == '"' -> "String"
i >= 0 && text[i] == ']' -> "List"
i >= 0 && text[i] == '}' -> "Dict"
i >= 0 && text[i] == ')' -> {
// Parenthesized expression: walk back to matching '(' and inspect the inner expression
var j = i - 1
var depth = 0
while (j >= 0) {
when (text[j]) {
')' -> depth++
'(' -> if (depth == 0) break else depth--
}
j--
}
if (j >= 0 && text[j] == '(') {
val innerS = (j + 1).coerceAtLeast(0)
val innerE = i.coerceAtMost(text.length)
if (innerS < innerE) {
val inner = text.substring(innerS, innerE).trim()
when {
inner.startsWith('"') && inner.endsWith('"') -> "String"
inner.startsWith('[') && inner.endsWith(']') -> "List"
inner.startsWith('{') && inner.endsWith('}') -> "Dict"
else -> null
}
} else null
} else null
}
else -> DocLookupUtils.guessClassFromCallBefore(text, dotPos, importedModules)
}
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: memberCtx dotPos=${dotPos} chBeforeDot='${if (dotPos>0) text[dotPos-1] else ' '}' classGuess=${className} imports=${importedModules}")
if (className != null) {
DocLookupUtils.resolveMemberWithInheritance(importedModules, className, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] QuickDoc: literal/call '$ident' resolved to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
log.info("[LYNG_DEBUG] QuickDoc: resolve failed for ${className}.${ident}")
}
}
}
// 4) As a fallback, if the caret is on an identifier text that matches any declaration name, show that
if (haveMini) mini.declarations.firstOrNull { it.name == ident }?.let {
log.info("[LYNG_DEBUG] QuickDoc: fallback by name '${it.name}' kind=${it::class.simpleName}")
return renderDeclDoc(it)
}
// 4) Consult BuiltinDocRegistry for imported modules (top-level and class members)
// Canonicalize import names using ImportManager, as users may write shortened names (e.g., "io.fs")
var importedModules = if (haveMini) DocLookupUtils.canonicalImportedModules(mini) else emptyList()
// If MiniAst failed or captured no imports, try a lightweight textual import scan
if (importedModules.isEmpty()) {
val fromText = extractImportsFromText(text)
if (fromText.isNotEmpty()) {
importedModules = fromText
}
}
// Always include stdlib as a fallback context
if (!importedModules.contains("lyng.stdlib")) importedModules = importedModules + "lyng.stdlib"
// 4a) try top-level decls
importedModules.forEach { mod ->
val docs = BuiltinDocRegistry.docsForModule(mod)
val matches = docs.filterIsInstance<MiniFunDecl>().filter { it.name == ident }
if (matches.isNotEmpty()) {
// Prefer overload by arity when caret is in a call position; otherwise show first
val arity = callArity(text, idRange.endOffset)
val chosen = arity?.let { a -> matches.firstOrNull { it.params.size == a } } ?: matches.first()
// If multiple and none matched arity, consider showing an overloads list
if (arity != null && chosen.params.size != arity && matches.size > 1) {
return renderOverloads(ident, matches)
}
return renderDeclDoc(chosen)
}
// Also allow values/consts
docs.filterIsInstance<MiniValDecl>().firstOrNull { it.name == ident }?.let { return renderDeclDoc(it) }
// And classes
docs.filterIsInstance<MiniClassDecl>().firstOrNull { it.name == ident }?.let { return renderDeclDoc(it) }
}
// Defensive fallback: if nothing found and it's a well-known stdlib function, render minimal inline docs
if (ident == "println" || ident == "print") {
val fallback = if (ident == "println")
"Print values to the standard output and append a newline. Accepts any number of arguments." else
"Print values to the standard output without a trailing newline. Accepts any number of arguments."
val title = "function $ident(values)"
return "<div class='doc-title'>${htmlEscape(title)}</div>" + styledMarkdown(htmlEscape(fallback))
}
// 4b) try class members like ClassName.member with inheritance fallback
val lhs = previousWordBefore(text, idRange.startOffset)
if (lhs != null && hasDotBetween(text, lhs.endOffset, idRange.startOffset)) {
val className = text.substring(lhs.startOffset, lhs.endOffset)
DocLookupUtils.resolveMemberWithInheritance(importedModules, className, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Inheritance resolved $className.$ident to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
} else {
// Heuristics when LHS is not an identifier (literals or call results):
// - List literal like [..].member → assume class List
// - Otherwise, try to find a unique class across imported modules that defines this member
val dotPos = findDotLeft(text, idRange.startOffset)
if (dotPos != null) {
val guessed = when {
looksLikeListLiteralBefore(text, dotPos) -> "List"
else -> DocLookupUtils.guessClassFromCallBefore(text, dotPos, importedModules)
}
if (guessed != null) {
DocLookupUtils.resolveMemberWithInheritance(importedModules, guessed, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Heuristic '$guessed.$ident' resolved via inheritance to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
} else {
// Extra fallback: try a small set of known receiver classes (covers literals when guess failed)
run {
val candidates = listOf("String", "Iterable", "Iterator", "List", "Collection", "Array", "Dict", "Regex")
for (c in candidates) {
DocLookupUtils.resolveMemberWithInheritance(importedModules, c, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Candidate '$c.$ident' resolved via inheritance to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
}
}
// As a last resort try aggregated String members (extensions from stdlib text)
run {
val classes = DocLookupUtils.aggregateClasses(importedModules)
val stringCls = classes["String"]
val m = stringCls?.members?.firstOrNull { it.name == ident }
if (m != null) {
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Aggregated fallback resolved String.$ident")
return when (m) {
is MiniMemberFunDecl -> renderMemberFunDoc("String", m)
is MiniMemberValDecl -> renderMemberValDoc("String", m)
}
}
}
// Search across classes; prefer Iterable, then Iterator, then List for common ops
DocLookupUtils.findMemberAcrossClasses(importedModules, ident)?.let { (owner, member) ->
if (DEBUG_INHERITANCE) log.info("[LYNG_DEBUG] Cross-class '$ident' resolved to $owner.${member.name}")
return when (member) {
is MiniMemberFunDecl -> renderMemberFunDoc(owner, member)
is MiniMemberValDecl -> renderMemberValDoc(owner, member)
}
}
}
}
}
if (DEBUG_LOG) log.info("[LYNG_DEBUG] QuickDoc: nothing found for ident='$ident'")
return null
}
/**
* Very lenient import extractor for cases when MiniAst is unavailable.
* Looks for lines like `import xxx.yyy` and returns canonical module names
* (prefixing with `lyng.` if missing).
*/
private fun extractImportsFromText(text: String): List<String> {
val result = LinkedHashSet<String>()
val re = Regex("(?m)^\\s*import\\s+([a-zA-Z_][a-zA-Z0-9_]*(?:\\.[a-zA-Z_][a-zA-Z0-9_]*)*)")
re.findAll(text).forEach { m ->
val raw = m.groupValues.getOrNull(1)?.trim().orEmpty()
if (raw.isNotEmpty()) {
val canon = if (raw.startsWith("lyng.")) raw else "lyng.$raw"
result.add(canon)
}
}
return result.toList()
}
// External docs registrars discovery via reflection to avoid hard dependencies on optional modules
private val externalDocsLoaded: Boolean by lazy { tryLoadExternalDocs() }
private fun ensureExternalDocsRegistered() { @Suppress("UNUSED_EXPRESSION") externalDocsLoaded }
private fun tryLoadExternalDocs(): Boolean {
return try {
// Try known registrars; ignore failures if module is absent
val cls = Class.forName("net.sergeych.lyngio.docs.FsBuiltinDocs")
val m = cls.getMethod("ensure")
m.invoke(null)
log.info("[LYNG_DEBUG] QuickDoc: external docs loaded: net.sergeych.lyngio.docs.FsBuiltinDocs.ensure() OK")
true
} catch (_: Throwable) {
// Seed a minimal plugin-local fallback so Path docs still work without lyngio
val seeded = try {
FsDocsFallback.ensureOnce()
} catch (_: Throwable) { false }
if (seeded) {
log.info("[LYNG_DEBUG] QuickDoc: external docs NOT found; seeded plugin fallback for lyng.io.fs")
} else {
log.info("[LYNG_DEBUG] QuickDoc: external docs NOT found (lyngio absent on classpath)")
}
seeded
}
}
override fun getCustomDocumentationElement(
editor: Editor,
file: PsiFile,
contextElement: PsiElement?,
targetOffset: Int
): PsiElement? {
// Ensure our provider gets a chance for Lyng files regardless of PSI sophistication
if (file.language != LyngLanguage) return null
return contextElement ?: file.findElementAt(targetOffset)
}
private fun renderDeclDoc(d: MiniDecl): String {
val title = when (d) {
is MiniFunDecl -> "function ${d.name}${signatureOf(d)}"
is MiniClassDecl -> "class ${d.name}"
is MiniValDecl -> if (d.mutable) "var ${d.name}${typeOf(d.type)}" else "val ${d.name}${typeOf(d.type)}"
else -> d.name
}
// Show full detailed documentation, not just the summary
val raw = d.doc?.raw
val doc: String? = if (raw.isNullOrBlank()) null else MarkdownRenderer.render(raw)
val sb = StringBuilder()
sb.append("<div class='doc-title'>").append(htmlEscape(title)).append("</div>")
if (!doc.isNullOrBlank()) sb.append(styledMarkdown(doc!!))
return sb.toString()
}
private fun renderParamDoc(fn: MiniFunDecl, p: MiniParam): String {
val title = "parameter ${p.name}${typeOf(p.type)} in ${fn.name}${signatureOf(fn)}"
return "<div class='doc-title'>${htmlEscape(title)}</div>"
}
private fun renderMemberFunDoc(className: String, m: MiniMemberFunDecl): String {
val params = m.params.joinToString(", ") { p ->
val ts = typeOf(p.type)
if (ts.isNotBlank()) "${p.name}${ts}" else p.name
}
val ret = typeOf(m.returnType)
val staticStr = if (m.isStatic) "static " else ""
val title = "${staticStr}method $className.${m.name}(${params})${ret}"
val raw = m.doc?.raw
val doc: String? = if (raw.isNullOrBlank()) null else MarkdownRenderer.render(raw)
val sb = StringBuilder()
sb.append("<div class='doc-title'>").append(htmlEscape(title)).append("</div>")
if (!doc.isNullOrBlank()) sb.append(styledMarkdown(doc!!))
return sb.toString()
}
private fun renderMemberValDoc(className: String, m: MiniMemberValDecl): String {
val ts = typeOf(m.type)
val kind = if (m.mutable) "var" else "val"
val staticStr = if (m.isStatic) "static " else ""
val title = "${staticStr}${kind} $className.${m.name}${ts}"
val raw = m.doc?.raw
val doc: String? = if (raw.isNullOrBlank()) null else MarkdownRenderer.render(raw)
val sb = StringBuilder()
sb.append("<div class='doc-title'>").append(htmlEscape(title)).append("</div>")
if (!doc.isNullOrBlank()) sb.append(styledMarkdown(doc!!))
return sb.toString()
}
private fun typeOf(t: MiniTypeRef?): String = when (t) {
is MiniTypeName -> ": ${t.segments.joinToString(".") { it.name }}${if (t.nullable) "?" else ""}"
is MiniGenericType -> {
val base = typeOf(t.base).removePrefix(": ")
val args = t.args.joinToString(", ") { typeOf(it).removePrefix(": ") }
": ${base}<${args}>${if (t.nullable) "?" else ""}"
}
is MiniFunctionType -> ": (..) -> ..${if (t.nullable) "?" else ""}"
is MiniTypeVar -> ": ${t.name}${if (t.nullable) "?" else ""}"
null -> ""
}
private fun signatureOf(fn: MiniFunDecl): String {
val params = fn.params.joinToString(", ") { p ->
val ts = typeOf(p.type)
if (ts.isNotBlank()) "${p.name}${ts}" else p.name
}
val ret = typeOf(fn.returnType)
return "(${params})${ret}"
}
private fun htmlEscape(s: String): String = buildString(s.length) {
for (ch in s) append(
when (ch) {
'<' -> "&lt;"
'>' -> "&gt;"
'&' -> "&amp;"
'"' -> "&quot;"
else -> ch
}
)
}
private fun styledMarkdown(html: String): String {
// IntelliJ doc renderer sanitizes and may surface <style> content as text.
// Strip any style tags defensively and keep markup lean; rely on platform defaults.
val safe = stripStyleTags(html)
return """
<div class="lyng-doc-md" style="max-width:72ch; line-height:1.4; font-size:0.96em;">
$safe
</div>
""".trimIndent()
}
private fun stripStyleTags(src: String): String {
// Remove any <style>...</style> blocks (case-insensitive, dotall)
val styleRegex = Regex("(?is)<style[^>]*>.*?</style>")
return src.replace(styleRegex, "")
}
// --- Simple helpers to support overload selection and heuristics ---
/**
* If identifier at [rightAfterIdent] is followed by a call like `(a, b)`,
* return the argument count; otherwise return null. Nested parentheses are
* handled conservatively to skip commas inside lambdas/parentheses.
*/
private fun callArity(text: String, rightAfterIdent: Int): Int? {
var i = rightAfterIdent
// Skip whitespace
while (i < text.length && text[i].isWhitespace()) i++
if (i >= text.length || text[i] != '(') return null
i++
var depth = 0
var commas = 0
var hasToken = false
while (i < text.length) {
val ch = text[i]
when (ch) {
'(' -> { depth++; hasToken = true }
')' -> {
if (depth == 0) {
// Empty parentheses => arity 0 if no token and no commas
if (!hasToken && commas == 0) return 0
return commas + 1
} else depth--
}
',' -> if (depth == 0) { commas++; hasToken = false }
'\n' -> {}
else -> if (!ch.isWhitespace()) hasToken = true
}
i++
}
return null
}
private fun renderOverloads(name: String, overloads: List<MiniFunDecl>): String {
val sb = StringBuilder()
sb.append("<div class='doc-title'>Overloads for ").append(htmlEscape(name)).append("</div>")
sb.append("<ul>")
overloads.forEach { fn ->
sb.append("<li><code>")
.append(htmlEscape("fun ${fn.name}${signatureOf(fn)}"))
.append("</code>")
fn.doc?.summary?.let { sum -> sb.append(" — ").append(htmlEscape(sum)) }
sb.append("</li>")
}
sb.append("</ul>")
return sb.toString()
}
private fun wordRangeAt(text: String, offset: Int): TextRange? {
if (text.isEmpty()) return null
var s = offset.coerceIn(0, text.length)
var e = s
while (s > 0 && isIdentChar(text[s - 1])) s--
while (e < text.length && isIdentChar(text[e])) e++
return if (e > s) TextRange(s, e) else null
}
private fun previousWordBefore(text: String, offset: Int): TextRange? {
// skip spaces and dots to the left, but stop after hitting a non-identifier or dot boundary
var i = (offset - 1).coerceAtLeast(0)
// first, move left past spaces
while (i > 0 && text[i].isWhitespace()) i--
// remember position to check for dot between words
val end = i + 1
// now find the start of the identifier
while (i >= 0 && isIdentChar(text[i])) i--
val start = (i + 1)
return if (start < end && start >= 0) TextRange(start, end) else null
}
private fun hasDotBetween(text: String, leftEnd: Int, rightStart: Int): Boolean {
val s = leftEnd.coerceAtLeast(0)
val e = rightStart.coerceAtMost(text.length)
if (e <= s) return false
for (i in s until e) if (text[i] == '.') return true
return false
}
private fun isIdentChar(c: Char): Boolean = c == '_' || c.isLetterOrDigit()
// --- Helpers for inheritance-aware and heuristic member lookup ---
// Removed: member/class resolution helpers moved to lynglib DocLookupUtils for reuse
private fun findDotLeft(text: String, rightStart: Int): Int? {
var i = (rightStart - 1).coerceAtLeast(0)
while (i >= 0 && text[i].isWhitespace()) i--
while (i >= 0) {
val ch = text[i]
if (ch == '.') return i
if (ch == '\n') return null
i--
}
return null
}
private fun looksLikeListLiteralBefore(text: String, dotPos: Int): Boolean {
// Look left for a closing ']' possibly with spaces, then a matching '[' before a comma or assignment
var i = (dotPos - 1).coerceAtLeast(0)
while (i >= 0 && text[i].isWhitespace()) i--
if (i < 0 || text[i] != ']') return false
var depth = 0
i--
while (i >= 0) {
val ch = text[i]
when (ch) {
']' -> depth++
'[' -> if (depth == 0) return true else depth--
'\n' -> return false
}
i--
}
return false
}
// Removed: guessClassFromCallBefore moved to DocLookupUtils
}
@@ -0,0 +1,21 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.docs
// 243: We do not use DocumentationTarget API here. Quick Docs works via
// AbstractDocumentationProvider registered as lang.documentationProvider.
internal object LyngDocumentationTargetsPlaceholder
@@ -0,0 +1,64 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.docs
import com.vladsch.flexmark.ext.autolink.AutolinkExtension
import com.vladsch.flexmark.ext.gfm.strikethrough.StrikethroughExtension
import com.vladsch.flexmark.ext.tables.TablesExtension
import com.vladsch.flexmark.html.HtmlRenderer
import com.vladsch.flexmark.parser.Parser
import com.vladsch.flexmark.util.data.MutableDataSet
/**
* Rich Markdown renderer for the IDEA Quick Docs using Flexmark.
*
* - Supports fenced code blocks (with language class "language-xyz")
* - Autolinks, tables, strikethrough
* - Converts soft breaks to <br/>
* - Tiny in-memory cache to avoid repeated parsing of the same doc blocks
*/
object MarkdownRenderer {
private val options = MutableDataSet().apply {
set(Parser.EXTENSIONS, listOf(
AutolinkExtension.create(),
TablesExtension.create(),
StrikethroughExtension.create(),
))
// Add CSS class for code fences like ```lyng → class="language-lyng"
set(HtmlRenderer.FENCED_CODE_LANGUAGE_CLASS_PREFIX, "language-")
// Treat single newlines as a space (soft break) so consecutive lines remain one paragraph.
// Real paragraph breaks require an empty line, hard breaks still work via Markdown (two spaces + \n).
set(HtmlRenderer.SOFT_BREAK, " ")
}
private val parser: Parser = Parser.builder(options).build()
private val renderer: HtmlRenderer = HtmlRenderer.builder(options).build()
private val cache = object : LinkedHashMap<String, String>(256, 0.75f, true) {
override fun removeEldestEntry(eldest: MutableMap.MutableEntry<String, String>?): Boolean = size > 256
}
fun render(markdown: String): String {
// Fast path: cache
synchronized(cache) { cache[markdown]?.let { return it } }
val node = parser.parse(markdown)
val html = renderer.render(node)
synchronized(cache) { cache[markdown] = html }
return html
}
}
@@ -0,0 +1,36 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.codeInsight.editorActions.BackspaceHandlerDelegate
import com.intellij.openapi.editor.Editor
import com.intellij.psi.PsiFile
/**
* Backspace handler (currently inactive, not registered). Minimal stub to keep build green.
* We will enable and implement smart behavior after API verification on the target IDE.
*/
class LyngBackspaceHandler : BackspaceHandlerDelegate() {
override fun beforeCharDeleted(c: Char, file: PsiFile, editor: Editor) {
// no-op
}
override fun charDeleted(c: Char, file: PsiFile, editor: Editor): Boolean {
// no-op; let default behavior stand
return false
}
}
@@ -0,0 +1,23 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
/**
* Disabled placeholder. We currently use the EditorPaste action handler (LyngPasteHandler)
* which works across IDE builds without relying on RawText API.
*/
class LyngCopyPastePreProcessorDisabled
@@ -0,0 +1,23 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
/**
* Placeholder for 2024.3+ RawText-based CopyPastePreProcessor.
* Not compiled against current SDK classpath; kept for future activation.
*/
class LyngCopyPastePreProcessor243Disabled
@@ -0,0 +1,204 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.application.options.CodeStyle
import com.intellij.codeInsight.editorActions.enter.EnterHandlerDelegate
import com.intellij.codeInsight.editorActions.enter.EnterHandlerDelegate.Result
import com.intellij.openapi.actionSystem.DataContext
import com.intellij.openapi.command.WriteCommandAction
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Caret
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.editor.actionSystem.EditorActionHandler
import com.intellij.openapi.project.Project
import com.intellij.openapi.util.Ref
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiDocumentManager
import com.intellij.psi.PsiFile
import com.intellij.psi.codeStyle.CodeStyleManager
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
class LyngEnterHandler : EnterHandlerDelegate {
private val log = Logger.getInstance(LyngEnterHandler::class.java)
override fun preprocessEnter(
file: PsiFile,
editor: Editor,
caretOffset: Ref<Int>,
caretAdvance: Ref<Int>,
dataContext: DataContext,
originalHandler: EditorActionHandler?
): Result {
if (file.language != LyngLanguage) return Result.Continue
if (log.isDebugEnabled) log.debug("[LyngEnter] preprocess in Lyng file at caretOffset=${caretOffset.get()}")
// Let the platform insert the newline; we will fix indentation in postProcessEnter.
return Result.Continue
}
override fun postProcessEnter(file: PsiFile, editor: Editor, dataContext: DataContext): Result {
if (file.language != LyngLanguage) return Result.Continue
val project = file.project
val doc = editor.document
val psiManager = PsiDocumentManager.getInstance(project)
psiManager.commitDocument(doc)
// Handle all carets independently to keep multi-caret scenarios sane
for (caret in editor.caretModel.allCarets) {
val line = doc.safeLineNumber(caret.offset)
if (line < 0) continue
if (log.isDebugEnabled) log.debug("[LyngEnter] postProcess at line=$line offset=${caret.offset}")
// Adjust previous '}' line if applicable, then indent the current line using our rules
adjustBraceAndCurrentIndent(project, file, doc, line)
// After indenting, ensure caret sits after the computed indent of this new line
moveCaretToIndentIfOnLeadingWs(editor, doc, file, line, caret)
}
return Result.Continue
}
private fun adjustBraceAndCurrentIndent(project: Project, file: PsiFile, doc: Document, currentLine: Int) {
val prevLine = currentLine - 1
if (prevLine >= 0) {
val prevText = doc.getLineText(prevLine)
val trimmed = prevText.trimStart()
// consider only code part before // comment
val code = trimmed.substringBefore("//").trim()
if (code == "}") {
// Previously we reindented the enclosed block on Enter after a lone '}'.
// Per new behavior, this action is now bound to typing '}' instead.
// Keep Enter flow limited to indenting the new line only.
}
}
// Adjust indent for the current (new) line
val currentStart = doc.getLineStartOffsetSafe(currentLine)
val csm = CodeStyleManager.getInstance(project)
csm.adjustLineIndent(file, currentStart)
// Fallback: if the platform didn't physically insert indentation, compute it from our formatter and apply
val lineStart = doc.getLineStartOffset(currentLine)
val lineEnd = doc.getLineEndOffset(currentLine)
val desiredIndent = computeDesiredIndent(project, doc, currentLine)
val firstNonWs = findFirstNonWs(doc, lineStart, lineEnd)
val currentIndentLen = firstNonWs - lineStart
if (desiredIndent.isNotEmpty() || currentIndentLen != 0) {
// Replace existing leading whitespace to match desired indent exactly
val replaceFrom = lineStart
val replaceTo = lineStart + currentIndentLen
if (doc.getText(TextRange(replaceFrom, replaceTo)) != desiredIndent) {
com.intellij.openapi.command.WriteCommandAction.runWriteCommandAction(project) {
doc.replaceString(replaceFrom, replaceTo, desiredIndent)
}
PsiDocumentManager.getInstance(project).commitDocument(doc)
if (log.isDebugEnabled) {
val dbg = desiredIndent.replace("\t", "\\t")
log.debug("[LyngEnter] rewrote current line leading WS to '$dbg' at line=$currentLine")
}
}
}
}
private fun reindentClosedBlockAroundBrace(project: Project, file: PsiFile, doc: Document, braceLine: Int) {
// Find the absolute index of the '}' at or before end of braceLine
val braceLineStart = doc.getLineStartOffset(braceLine)
val braceLineEnd = doc.getLineEndOffset(braceLine)
val rawBraceLine = doc.getText(TextRange(braceLineStart, braceLineEnd))
val codeBraceLine = rawBraceLine.substringBefore("//")
val closeRel = codeBraceLine.lastIndexOf('}')
if (closeRel < 0) return
val closeAbs = braceLineStart + closeRel
// Compute the enclosing block range in raw text (document char sequence)
val blockRange = net.sergeych.lyng.format.BraceUtils.findEnclosingBlockRange(doc.charsSequence, closeAbs, includeTrailingNewline = true)
?: return
val options = CodeStyle.getIndentOptions(project, doc)
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
// Run partial reindent over the slice and replace only if changed
val whole = doc.text
val updated = LyngFormatter.reindentRange(whole, blockRange, cfg, preserveBaseIndent = true)
if (updated != whole) {
WriteCommandAction.runWriteCommandAction(project) {
doc.replaceString(0, doc.textLength, updated)
}
PsiDocumentManager.getInstance(project).commitDocument(doc)
if (log.isDebugEnabled) log.debug("[LyngEnter] reindented closed block range=${'$'}blockRange")
}
}
private fun moveCaretToIndentIfOnLeadingWs(editor: Editor, doc: Document, file: PsiFile, line: Int, caret: Caret) {
if (line < 0 || line >= doc.lineCount) return
val lineStart = doc.getLineStartOffset(line)
val lineEnd = doc.getLineEndOffset(line)
val desiredIndent = computeDesiredIndent(file.project, doc, line)
val firstNonWs = (lineStart + desiredIndent.length).coerceAtMost(doc.textLength)
val caretOffset = caret.offset
// If caret is at beginning of the line or still within the leading whitespace, move it after indent
val target = firstNonWs.coerceIn(lineStart, lineEnd)
caret.moveToOffset(target)
}
private fun computeDesiredIndent(project: Project, doc: Document, line: Int): String {
val options = CodeStyle.getIndentOptions(project, doc)
val start = 0
val end = doc.getLineEndOffset(line)
val snippet = doc.getText(TextRange(start, end))
val isBlankLine = doc.getLineText(line).trim().isEmpty()
val snippetForCalc = if (isBlankLine) snippet + "x" else snippet
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
val formatted = LyngFormatter.reindent(snippetForCalc, cfg)
val lastNl = formatted.lastIndexOf('\n')
val lastLine = if (lastNl >= 0) formatted.substring(lastNl + 1) else formatted
val wsLen = lastLine.indexOfFirst { it != ' ' && it != '\t' }.let { if (it < 0) lastLine.length else it }
return lastLine.substring(0, wsLen)
}
private fun findFirstNonWs(doc: Document, start: Int, end: Int): Int {
var i = start
val text = doc.charsSequence
while (i < end) {
val ch = text[i]
if (ch != ' ' && ch != '\t') break
i++
}
return i
}
private fun Document.safeLineNumber(offset: Int): Int =
getLineNumber(offset.coerceIn(0, textLength))
private fun Document.getLineText(line: Int): String {
if (line < 0 || line >= lineCount) return ""
val start = getLineStartOffset(line)
val end = getLineEndOffset(line)
return getText(TextRange(start, end))
}
private fun Document.getLineStartOffsetSafe(line: Int): Int =
if (line < 0) 0 else getLineStartOffset(line.coerceAtMost(lineCount - 1).coerceAtLeast(0))
}
@@ -0,0 +1,22 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
/**
* Disabled placeholder: we rely on LyngPasteHandler (EditorPaste action handler).
*/
class LyngOnPasteProcessorDisabled
@@ -0,0 +1,166 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.application.options.CodeStyle
import com.intellij.openapi.actionSystem.DataContext
import com.intellij.openapi.command.WriteCommandAction
import com.intellij.openapi.editor.Caret
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.editor.actionSystem.EditorWriteActionHandler
import com.intellij.openapi.ide.CopyPasteManager
import com.intellij.psi.PsiDocumentManager
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import java.awt.datatransfer.DataFlavor
/**
* Smart Paste using an editor action handler (avoids RawText API variance).
* Reindents pasted blocks when caret is in leading whitespace and the setting is enabled.
*/
class LyngPasteHandler : EditorWriteActionHandler(true) {
private val log = com.intellij.openapi.diagnostic.Logger.getInstance(LyngPasteHandler::class.java)
override fun doExecute(editor: Editor, caret: Caret?, dataContext: DataContext) {
val project = editor.project
if (project == null) return
val psiDocMgr = PsiDocumentManager.getInstance(project)
val file = psiDocMgr.getPsiFile(editor.document)
if (file == null || file.language != LyngLanguage) {
pasteAsIs(editor)
return
}
val settings = LyngFormatterSettings.getInstance(project)
if (!settings.reindentPastedBlocks) {
pasteAsIs(editor)
return
}
val text = CopyPasteManager.getInstance().getContents<String>(DataFlavor.stringFlavor)
if (text == null) {
pasteAsIs(editor)
return
}
val caretModel = editor.caretModel
val effectiveCaret = caret ?: caretModel.currentCaret
val doc = editor.document
PsiDocumentManager.getInstance(project).commitDocument(doc)
// Paste the text as-is first, then compute the inserted range and reindent that slice
val options = CodeStyle.getIndentOptions(project, doc)
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
// Replace selection (if any) or insert at caret with original clipboard text
val selModel = editor.selectionModel
val replaceStart = if (selModel.hasSelection()) selModel.selectionStart else effectiveCaret.offset
val replaceEnd = if (selModel.hasSelection()) selModel.selectionEnd else effectiveCaret.offset
WriteCommandAction.runWriteCommandAction(project) {
log.info("[LyngPaste] handler invoked for Lyng file; setting ON=${settings.reindentPastedBlocks}")
// Step 1: paste as-is
val beforeLen = doc.textLength
doc.replaceString(replaceStart, replaceEnd, text)
psiDocMgr.commitDocument(doc)
// Step 2: compute the freshly inserted range robustly (account for line-separator normalization)
val insertedStart = replaceStart
val delta = doc.textLength - beforeLen + (replaceEnd - replaceStart)
val insertedEndExclusive = (insertedStart + delta).coerceIn(insertedStart, doc.textLength)
// Expand to full lines to let the formatter compute proper base/closing alignment
val lineStart = run {
var i = (insertedStart - 1).coerceAtLeast(0)
while (i >= 0 && doc.charsSequence[i] != '\n') i--
i + 1
}
var lineEndInclusive = run {
var i = insertedEndExclusive
val seq = doc.charsSequence
while (i < seq.length && seq[i] != '\n') i++
// include trailing newline if present
if (i < seq.length && seq[i] == '\n') i + 1 else i
}
// If the next non-whitespace char right after the insertion is a closing brace '}',
// include that brace line into the formatting slice for better block alignment.
run {
val seq = doc.charsSequence
var j = insertedEndExclusive
while (j < seq.length && (seq[j] == ' ' || seq[j] == '\t' || seq[j] == '\n' || seq[j] == '\r')) j++
if (j < seq.length && seq[j] == '}') {
var k = j
while (k < seq.length && seq[k] != '\n') k++
lineEndInclusive = if (k < seq.length && seq[k] == '\n') k + 1 else k
}
}
val fullTextBefore = doc.text
val expandedRange = (lineStart until lineEndInclusive)
log.info("[LyngPaste] inserted=[$insertedStart,$insertedEndExclusive) expanded=[$lineStart,$lineEndInclusive)")
val updatedFull = LyngFormatter.reindentRange(
fullTextBefore,
expandedRange,
cfg,
preserveBaseIndent = true,
baseIndentFrom = insertedStart
)
if (updatedFull != fullTextBefore) {
val delta = updatedFull.length - fullTextBefore.length
doc.replaceString(0, doc.textLength, updatedFull)
psiDocMgr.commitDocument(doc)
caretModel.moveToOffset((insertedEndExclusive + delta).coerceIn(0, doc.textLength))
log.info("[LyngPaste] applied reindent to expanded range")
} else {
// No changes after reindent — just move caret to end of the inserted text
caretModel.moveToOffset(insertedEndExclusive)
log.info("[LyngPaste] no changes after reindent")
}
selModel.removeSelection()
}
}
private fun pasteAsIs(editor: Editor) {
val text = CopyPasteManager.getInstance().getContents<String>(DataFlavor.stringFlavor) ?: return
pasteText(editor, text)
}
private fun pasteText(editor: Editor, text: String) {
val project = editor.project ?: return
val doc = editor.document
val caretModel = editor.caretModel
val selModel = editor.selectionModel
WriteCommandAction.runWriteCommandAction(project) {
val replaceStart = if (selModel.hasSelection()) selModel.selectionStart else caretModel.offset
val replaceEnd = if (selModel.hasSelection()) selModel.selectionEnd else caretModel.offset
doc.replaceString(replaceStart, replaceEnd, text)
PsiDocumentManager.getInstance(project).commitDocument(doc)
caretModel.moveToOffset(replaceStart + text.length)
selModel.removeSelection()
}
}
// no longer used
}
@@ -0,0 +1,83 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.application.options.CodeStyle
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.project.Project
import com.intellij.psi.PsiDocumentManager
import com.intellij.psi.PsiFile
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
/**
* Helper for preparing reindented pasted text. EP wiring is deferred until API
* signature is finalized for the target IDE build.
*/
object LyngPastePreProcessor {
fun reindentForPaste(
project: Project,
editor: Editor,
file: PsiFile,
text: String
): String {
if (file.language != LyngLanguage) return text
val settings = LyngFormatterSettings.getInstance(project)
if (!settings.reindentPastedBlocks) return text
val doc = editor.document
PsiDocumentManager.getInstance(project).commitDocument(doc)
val options = CodeStyle.getIndentOptions(project, doc)
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
// Only apply smart paste when caret is in leading whitespace position of its line
val caret = editor.caretModel.currentCaret
val line = doc.getLineNumber(caret.offset.coerceIn(0, doc.textLength))
if (line < 0 || line >= doc.lineCount) return text
val lineStart = doc.getLineStartOffset(line)
val firstNonWs = firstNonWhitespace(doc, lineStart, doc.getLineEndOffset(line))
if (caret.offset > firstNonWs) return text
val baseIndent = doc.charsSequence.subSequence(lineStart, caret.offset).toString()
val reindented = LyngFormatter.reindent(text, cfg)
// Prefix each non-empty line with base indent to preserve surrounding indentation
val lines = reindented.split('\n')
val sb = StringBuilder(reindented.length + lines.size * baseIndent.length)
for ((idx, ln) in lines.withIndex()) {
if (ln.isNotEmpty()) sb.append(baseIndent).append(ln) else sb.append(ln)
if (idx < lines.lastIndex) sb.append('\n')
}
return sb.toString()
}
private fun firstNonWhitespace(doc: com.intellij.openapi.editor.Document, from: Int, to: Int): Int {
val seq = doc.charsSequence
var i = from
while (i < to) {
val ch = seq[i]
if (ch != ' ' && ch != '\t') break
i++
}
return i
}
}
@@ -0,0 +1,38 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.project.Project
import com.intellij.psi.PsiDocumentManager
import com.intellij.psi.PsiFile
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
/**
* Smart Paste helper for Lyng. Not registered as EP yet to keep build stable across IDE SDKs.
* Use `processOnPasteIfEnabled` from a CopyPastePreProcessor adapter once API signature is finalized.
*/
object LyngSmartPastePreProcessorHelper {
fun processOnPasteIfEnabled(project: Project, file: PsiFile, editor: Editor, text: String): String {
if (file.language != LyngLanguage) return text
val settings = LyngFormatterSettings.getInstance(project)
if (!settings.reindentPastedBlocks) return text
PsiDocumentManager.getInstance(project).commitDocument(editor.document)
return LyngPastePreProcessor.reindentForPaste(project, editor, file, text)
}
}
@@ -0,0 +1,103 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.editor
import com.intellij.application.options.CodeStyle
import com.intellij.codeInsight.editorActions.TypedHandlerDelegate
import com.intellij.openapi.command.WriteCommandAction
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.project.Project
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiDocumentManager
import com.intellij.psi.PsiFile
import com.intellij.psi.codeStyle.CodeStyleManager
import net.sergeych.lyng.format.BraceUtils
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
class LyngTypedHandler : TypedHandlerDelegate() {
private val log = Logger.getInstance(LyngTypedHandler::class.java)
override fun charTyped(c: Char, project: Project, editor: Editor, file: PsiFile): Result {
if (file.language != LyngLanguage) return Result.CONTINUE
if (c != '}') return Result.CONTINUE
val doc = editor.document
PsiDocumentManager.getInstance(project).commitDocument(doc)
val offset = editor.caretModel.offset
val line = doc.getLineNumber((offset - 1).coerceAtLeast(0))
if (line < 0) return Result.CONTINUE
val rawLine = doc.getLineText(line)
val code = rawLine.substringBefore("//").trim()
if (code == "}") {
val settings = LyngFormatterSettings.getInstance(project)
if (settings.reindentClosedBlockOnEnter) {
reindentClosedBlockAroundBrace(project, file, doc, line)
}
// After block reindent, adjust line indent to what platform thinks (no-op in many cases)
val lineStart = doc.getLineStartOffset(line)
CodeStyleManager.getInstance(project).adjustLineIndent(file, lineStart)
}
return Result.CONTINUE
}
private fun reindentClosedBlockAroundBrace(project: Project, file: PsiFile, doc: Document, braceLine: Int) {
val braceLineStart = doc.getLineStartOffset(braceLine)
val braceLineEnd = doc.getLineEndOffset(braceLine)
val rawBraceLine = doc.getText(TextRange(braceLineStart, braceLineEnd))
val codeBraceLine = rawBraceLine.substringBefore("//")
val closeRel = codeBraceLine.lastIndexOf('}')
if (closeRel < 0) return
val closeAbs = braceLineStart + closeRel
val blockRange = BraceUtils.findEnclosingBlockRange(
doc.charsSequence,
closeAbs,
includeTrailingNewline = true
) ?: return
val options = CodeStyle.getIndentOptions(project, doc)
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
val whole = doc.text
val updated = LyngFormatter.reindentRange(whole, blockRange, cfg, preserveBaseIndent = true)
if (updated != whole) {
WriteCommandAction.runWriteCommandAction(project) {
doc.replaceString(0, doc.textLength, updated)
}
PsiDocumentManager.getInstance(project).commitDocument(doc)
if (log.isDebugEnabled) log.debug("[LyngTyped] reindented closed block range=$blockRange")
}
}
private fun Document.getLineText(line: Int): String {
if (line < 0 || line >= lineCount) return ""
val start = getLineStartOffset(line)
val end = getLineEndOffset(line)
return getText(TextRange(start, end))
}
}
@@ -0,0 +1,58 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.format
import com.intellij.formatting.*
import com.intellij.lang.ASTNode
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiElement
import com.intellij.psi.PsiFile
import com.intellij.psi.codeStyle.CodeStyleSettings
/**
* Minimal formatting model: enables Reformat Code to at least re-apply indentation via LineIndentProvider
* and normalize whitespace. We don’t implement a full PSI-based tree yet; this block treats the whole file
* as a single formatting region and lets platform query line indents.
*/
class LyngFormattingModelBuilder : FormattingModelBuilder {
override fun createModel(element: PsiElement, settings: CodeStyleSettings): FormattingModel {
val file = element.containingFile
val rootBlock = LineBlocksRootBlock(file, settings)
return FormattingModelProvider.createFormattingModelForPsiFile(file, rootBlock, settings)
}
override fun getRangeAffectingIndent(file: PsiFile, offset: Int, elementAtOffset: ASTNode?): TextRange? = null
}
private class LineBlocksRootBlock(
private val file: PsiFile,
private val settings: CodeStyleSettings
) : Block {
override fun getTextRange(): TextRange = file.textRange
override fun getSubBlocks(): List<Block> = emptyList()
override fun getWrap(): Wrap? = null
override fun getIndent(): Indent? = Indent.getNoneIndent()
override fun getAlignment(): Alignment? = null
override fun getSpacing(child1: Block?, child2: Block): Spacing? = null
override fun getChildAttributes(newChildIndex: Int): ChildAttributes = ChildAttributes(Indent.getNoneIndent(), null)
override fun isIncomplete(): Boolean = false
override fun isLeaf(): Boolean = false
}
// Intentionally no sub-blocks/spacing: indentation is handled by PreFormatProcessor + LineIndentProvider
@@ -0,0 +1,103 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.format
import com.intellij.application.options.CodeStyle
import com.intellij.lang.Language
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiDocumentManager
import com.intellij.psi.codeStyle.CommonCodeStyleSettings.IndentOptions
import com.intellij.psi.codeStyle.lineIndent.LineIndentProvider
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
/**
* Lightweight indentation provider for Lyng.
*
* Rules (heuristic, text-based):
* - New lines after an opening brace/paren increase indent level.
* - Lines starting with a closing brace/paren decrease indent level by one.
* - Keeps previous non-empty line's indent as baseline otherwise.
*/
class LyngLineIndentProvider : LineIndentProvider {
override fun getLineIndent(project: com.intellij.openapi.project.Project, editor: Editor, language: Language?, offset: Int): String? {
if (language != null && language != LyngLanguage) return null
val doc = editor.document
PsiDocumentManager.getInstance(project).commitDocument(doc)
val options = CodeStyle.getIndentOptions(project, doc)
val line = doc.getLineNumberSafe(offset)
val indent = computeDesiredIndentFromCore(doc, line, options)
return indent
}
override fun isSuitableFor(language: Language?): Boolean = language == null || language == LyngLanguage
private fun Document.getLineNumberSafe(offset: Int): Int =
getLineNumber(offset.coerceIn(0, textLength))
private fun Document.getLineText(line: Int): String {
if (line < 0 || line >= lineCount) return ""
val start = getLineStartOffset(line)
val end = getLineEndOffset(line)
return getText(TextRange(start, end))
}
private fun indentUnit(options: IndentOptions): String =
if (options.USE_TAB_CHARACTER) "\t" else " ".repeat(options.INDENT_SIZE.coerceAtLeast(1))
private fun indentOfLine(doc: Document, line: Int): String {
val s = doc.getLineText(line)
val i = s.indexOfFirst { !it.isWhitespace() }
return if (i <= 0) s.takeWhile { it == ' ' || it == '\t' } else s.substring(0, i)
}
private fun countIndentUnits(indent: String, options: IndentOptions): Int {
if (indent.isEmpty()) return 0
if (options.USE_TAB_CHARACTER) return indent.count { it == '\t' }
val size = options.INDENT_SIZE.coerceAtLeast(1)
var spaces = 0
for (ch in indent) spaces += if (ch == '\t') size else 1
return spaces / size
}
private fun computeDesiredIndentFromCore(doc: Document, line: Int, options: IndentOptions): String {
// Build a minimal text consisting of all previous lines and the current line.
// Special case: when the current line is blank (newly created by Enter), compute the
// indent as if there was a non-whitespace character at line start (append a sentinel).
val start = 0
val end = doc.getLineEndOffset(line)
val snippet = doc.getText(TextRange(start, end))
val isBlankLine = doc.getLineText(line).trim().isEmpty()
val snippetForCalc = if (isBlankLine) snippet + "x" else snippet
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
val formatted = LyngFormatter.reindent(snippetForCalc, cfg)
// Grab the last line's leading whitespace as the indent for the current line
val lastNl = formatted.lastIndexOf('\n')
val lastLine = if (lastNl >= 0) formatted.substring(lastNl + 1) else formatted
val wsLen = lastLine.indexOfFirst { it != ' ' && it != '\t' }.let { if (it < 0) lastLine.length else it }
return lastLine.substring(0, wsLen)
}
}
@@ -0,0 +1,22 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.format
// Placeholder: we planned a post-format processor fallback, but the 2024.3 platform
// does not expose the older PostFormatProcessor API in our current dependency set.
// Reformat Code will use the registered lang.formatter + LineIndentProvider.
internal object LyngPostFormatProcessorPlaceholder
@@ -0,0 +1,159 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.format
import com.intellij.application.options.CodeStyle
import com.intellij.lang.ASTNode
import com.intellij.openapi.project.Project
import com.intellij.openapi.util.TextRange
import com.intellij.psi.codeStyle.CodeStyleManager
import com.intellij.psi.impl.source.codeStyle.PreFormatProcessor
import net.sergeych.lyng.format.LyngFormatConfig
import net.sergeych.lyng.format.LyngFormatter
import net.sergeych.lyng.idea.LyngLanguage
/**
* Idempotent indentation fixer executed by Reformat Code before formatting.
* It walks all lines in the affected range and applies exact indentation using
* CodeStyleManager.adjustLineIndent(), which delegates to our LineIndentProvider.
*/
class LyngPreFormatProcessor : PreFormatProcessor {
override fun process(element: ASTNode, range: TextRange): TextRange {
val file = element.psi?.containingFile ?: return range
if (file.language != LyngLanguage) return range
val project: Project = file.project
val doc = file.viewProvider.document ?: return range
val psiDoc = com.intellij.psi.PsiDocumentManager.getInstance(project)
val options = CodeStyle.getIndentOptions(project, doc)
val settings = net.sergeych.lyng.idea.settings.LyngFormatterSettings.getInstance(project)
// When both spacing and wrapping are OFF, still fix indentation for the whole file to
// guarantee visible changes on Reformat Code.
val runFullFileIndent = !settings.enableSpacing && !settings.enableWrapping
// Maintain a working range and a modification flag to avoid stale offsets after replacements
var modified = false
fun fullRange(): TextRange = TextRange(0, doc.textLength)
var workingRange: TextRange = range.intersection(fullRange()) ?: fullRange()
val startLine = if (runFullFileIndent) 0 else doc.getLineNumber(workingRange.startOffset)
val endLine = if (runFullFileIndent) (doc.lineCount - 1).coerceAtLeast(0)
else doc.getLineNumber(workingRange.endOffset.coerceAtMost(doc.textLength))
fun codePart(s: String): String {
val idx = s.indexOf("//")
return if (idx >= 0) s.substring(0, idx) else s
}
// Pre-scan to compute balances up to startLine
var blockLevel = 0
var parenBalance = 0
var bracketBalance = 0
for (ln in 0 until startLine) {
val text = doc.getText(TextRange(doc.getLineStartOffset(ln), doc.getLineEndOffset(ln)))
for (ch in codePart(text)) when (ch) {
'{' -> blockLevel++
'}' -> if (blockLevel > 0) blockLevel--
'(' -> parenBalance++
')' -> if (parenBalance > 0) parenBalance--
'[' -> bracketBalance++
']' -> if (bracketBalance > 0) bracketBalance--
}
}
// Re-indent each line deterministically (idempotent). We avoid any content
// rewriting here to prevent long-running passes or re-entrant formatting.
for (line in startLine..endLine) {
val lineStart = doc.getLineStartOffset(line)
// adjustLineIndent delegates to our LineIndentProvider which computes
// indentation from scratch; this is safe and idempotent
CodeStyleManager.getInstance(project).adjustLineIndent(file, lineStart)
// After indentation, update block/paren/bracket balances using the current line text
val lineEnd = doc.getLineEndOffset(line)
val text = doc.getText(TextRange(lineStart, lineEnd))
val code = codePart(text)
for (ch in code) when (ch) {
'{' -> blockLevel++
'}' -> if (blockLevel > 0) blockLevel--
'(' -> parenBalance++
')' -> if (parenBalance > 0) parenBalance--
'[' -> bracketBalance++
']' -> if (bracketBalance > 0) bracketBalance--
}
}
// If both spacing and wrapping are OFF, explicitly reindent the text using core formatter to
// guarantee indentation is fixed even when the platform doesn't rewrite whitespace by itself.
if (!settings.enableSpacing && !settings.enableWrapping) {
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
)
val full = fullRange()
val r = if (runFullFileIndent) full else workingRange.intersection(full) ?: full
val text = doc.getText(r)
val formatted = LyngFormatter.reindent(text, cfg)
if (formatted != text) {
doc.replaceString(r.startOffset, r.endOffset, formatted)
modified = true
psiDoc.commitDocument(doc)
workingRange = fullRange()
}
}
// Optionally apply spacing using the core formatter if enabled in settings (wrapping stays off)
if (settings.enableSpacing) {
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
applySpacing = true,
applyWrapping = false,
)
val safe = workingRange.intersection(fullRange()) ?: fullRange()
val text = doc.getText(safe)
val formatted = LyngFormatter.format(text, cfg)
if (formatted != text) {
doc.replaceString(safe.startOffset, safe.endOffset, formatted)
modified = true
psiDoc.commitDocument(doc)
workingRange = fullRange()
}
}
// Optionally apply wrapping (after spacing) when enabled
if (settings.enableWrapping) {
val cfg = LyngFormatConfig(
indentSize = options.INDENT_SIZE.coerceAtLeast(1),
useTabs = options.USE_TAB_CHARACTER,
continuationIndentSize = options.CONTINUATION_INDENT_SIZE.coerceAtLeast(options.INDENT_SIZE.coerceAtLeast(1)),
applySpacing = settings.enableSpacing,
applyWrapping = true,
)
val safe2 = workingRange.intersection(fullRange()) ?: fullRange()
val text2 = doc.getText(safe2)
val wrapped = LyngFormatter.format(text2, cfg)
if (wrapped != text2) {
doc.replaceString(safe2.startOffset, safe2.endOffset, wrapped)
modified = true
psiDoc.commitDocument(doc)
workingRange = fullRange()
}
}
// Return a safe range for the formatter to continue with, preventing stale offsets
return if (modified) fullRange() else (range.intersection(fullRange()) ?: fullRange())
}
}
@@ -0,0 +1,43 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.codeInsight.daemon.DaemonCodeAnalyzer
import com.intellij.codeInsight.intention.IntentionAction
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.project.Project
import com.intellij.psi.PsiFile
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
/**
* Lightweight quick-fix that adds a word to the per-project Lyng dictionary.
*/
class AddToLyngDictionaryFix(private val word: String) : IntentionAction {
override fun getText(): String = "Add '$word' to Lyng dictionary"
override fun getFamilyName(): String = "Lyng Spelling"
override fun isAvailable(project: Project, editor: Editor?, file: PsiFile?): Boolean = word.isNotBlank()
override fun startInWriteAction(): Boolean = true
override fun invoke(project: Project, editor: Editor?, file: PsiFile?) {
val settings = LyngFormatterSettings.getInstance(project)
val learned = settings.learnedWords
learned.add(word.lowercase())
settings.learnedWords = learned
// Restart daemon to refresh highlights
if (file != null) DaemonCodeAnalyzer.getInstance(project).restart(file)
}
}
@@ -0,0 +1,83 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.openapi.diagnostic.Logger
import java.io.BufferedReader
import java.io.InputStreamReader
import java.util.zip.GZIPInputStream
/**
* Very simple English dictionary loader for offline suggestions on IC-243.
* It loads a word list from classpath resources. Supports plain text (one word per line)
* and gzipped text if the resource ends with .gz.
*/
object EnglishDictionary {
private val log = Logger.getInstance(EnglishDictionary::class.java)
@Volatile private var loaded = false
@Volatile private var words: Set<String> = emptySet()
/**
* Load dictionary from bundled resources (once).
* If multiple candidates exist, the first found is used.
*/
private fun ensureLoaded() {
if (loaded) return
synchronized(this) {
if (loaded) return
val candidates = listOf(
// preferred large bundles first (add en-basic.txt.gz ~3–5MB here)
"/dictionaries/en-basic.txt.gz",
"/dictionaries/en-large.txt.gz",
// plain text fallbacks
"/dictionaries/en-basic.txt",
"/dictionaries/en-large.txt",
)
val merged = HashSet<String>(128_000)
for (res in candidates) {
try {
val stream = javaClass.getResourceAsStream(res) ?: continue
val reader = if (res.endsWith(".gz"))
BufferedReader(InputStreamReader(GZIPInputStream(stream)))
else
BufferedReader(InputStreamReader(stream))
var loadedCount = 0
reader.useLines { seq -> seq.forEach { line ->
val w = line.trim()
if (w.isNotEmpty() && !w.startsWith("#")) { merged += w.lowercase(); loadedCount++ }
} }
log.info("EnglishDictionary: loaded $loadedCount words from $res (total=${merged.size})")
} catch (t: Throwable) {
log.info("EnglishDictionary: failed to load $res: ${t.javaClass.simpleName}: ${t.message}")
}
}
if (merged.isEmpty()) {
// Fallback minimal set
merged += setOf("comment","comments","error","errors","found","file","not","word","words","count","value","name","class","function","string")
log.info("EnglishDictionary: using minimal built-in set (${merged.size})")
}
words = merged
loaded = true
}
}
fun allWords(): Set<String> {
ensureLoaded()
return words
}
}
@@ -0,0 +1,608 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* Grazie-backed annotator for Lyng files.
*
* It consumes the MiniAst-driven LyngSpellIndex and, when Grazie is present,
* tries to run Grazie checks on the extracted TextContent. Results are painted
* as warnings in the editor. If the Grazie API changes, we use reflection and
* fail softly with INFO logs (no errors shown to users).
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.codeInsight.daemon.DaemonCodeAnalyzer
import com.intellij.grazie.text.TextContent
import com.intellij.grazie.text.TextContent.TextDomain
import com.intellij.ide.plugins.PluginManagerCore
import com.intellij.lang.annotation.AnnotationHolder
import com.intellij.lang.annotation.ExternalAnnotator
import com.intellij.lang.annotation.HighlightSeverity
import com.intellij.openapi.application.ApplicationManager
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.colors.TextAttributesKey
import com.intellij.openapi.project.DumbAware
import com.intellij.openapi.util.Key
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiFile
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import net.sergeych.lyng.idea.spell.LyngSpellIndex
class LyngGrazieAnnotator : ExternalAnnotator<LyngGrazieAnnotator.Input, LyngGrazieAnnotator.Result>(), DumbAware {
private val log = Logger.getInstance(LyngGrazieAnnotator::class.java)
companion object {
// Cache GrammarChecker availability to avoid repeated reflection + noisy logs
@Volatile
private var grammarCheckerAvailable: Boolean? = null
@Volatile
private var grammarCheckerMissingLogged: Boolean = false
private fun isGrammarCheckerKnownMissing(): Boolean = (grammarCheckerAvailable == false)
private fun markGrammarCheckerMissingOnce(log: Logger, message: String) {
if (!grammarCheckerMissingLogged) {
// Downgrade to debug to reduce log noise across projects/sessions
log.debug(message)
grammarCheckerMissingLogged = true
}
}
private val RETRY_KEY: Key<Long> = Key.create("LYNG_GRAZIE_ANN_RETRY_STAMP")
}
data class Input(val modStamp: Long)
data class Finding(val range: TextRange, val message: String)
data class Result(val modStamp: Long, val findings: List<Finding>)
override fun collectInformation(file: PsiFile): Input? {
val doc: Document = file.viewProvider.document ?: return null
// Only require Grazie presence; index readiness is checked in apply with a retry.
val grazie = isGrazieInstalled()
if (!grazie) {
log.info("LyngGrazieAnnotator.collectInformation: skip (grazie=false) file='${file.name}'")
return null
}
log.info("LyngGrazieAnnotator.collectInformation: file='${file.name}', modStamp=${doc.modificationStamp}")
return Input(doc.modificationStamp)
}
override fun doAnnotate(collectedInfo: Input?): Result? {
// All heavy lifting is done in apply where we have the file context
return collectedInfo?.let { Result(it.modStamp, emptyList()) }
}
override fun apply(file: PsiFile, annotationResult: Result?, holder: AnnotationHolder) {
if (annotationResult == null || !isGrazieInstalled()) return
val doc = file.viewProvider.document ?: return
val idx = LyngSpellIndex.getUpToDate(file) ?: run {
log.info("LyngGrazieAnnotator.apply: index not ready for '${file.name}', scheduling one-shot restart")
scheduleOneShotRestart(file, annotationResult.modStamp)
return
}
val settings = LyngFormatterSettings.getInstance(file.project)
// Build TextContent fragments for comments/strings/identifiers according to settings
val fragments = mutableListOf<Pair<TextContent, TextRange>>()
try {
fun addFragments(ranges: List<TextRange>, domain: TextDomain) {
for (r in ranges) {
val local = rangeToTextContent(file, domain, r) ?: continue
fragments += local to r
}
}
// Comments always via COMMENTS
addFragments(idx.comments, TextDomain.COMMENTS)
// Strings: LITERALS if requested, else COMMENTS if fallback enabled
if (settings.spellCheckStringLiterals) {
val domain = if (settings.grazieTreatLiteralsAsComments) TextDomain.COMMENTS else TextDomain.LITERALS
addFragments(idx.strings, domain)
}
// Identifiers via COMMENTS to force painting in 243 unless user disables fallback
val idsDomain = if (settings.grazieTreatIdentifiersAsComments) TextDomain.COMMENTS else TextDomain.DOCUMENTATION
addFragments(idx.identifiers, idsDomain)
log.info(
"LyngGrazieAnnotator.apply: file='${file.name}', idxCounts ids=${idx.identifiers.size}, comments=${idx.comments.size}, strings=${idx.strings.size}, builtFragments=${fragments.size}"
)
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator: failed to build TextContent fragments: ${e.javaClass.simpleName}: ${e.message}")
return
}
if (fragments.isEmpty()) return
val findings = mutableListOf<Finding>()
var totalReturned = 0
var chosenEntry: String? = null
for ((content, hostRange) in fragments) {
try {
val (typos, entryNote) = runGrazieChecksWithTracing(file, content)
if (chosenEntry == null) chosenEntry = entryNote
if (typos != null) {
totalReturned += typos.size
for (t in typos) {
val rel = extractRangeFromTypo(t) ?: continue
// Map relative range inside fragment to host file range
val abs = TextRange(hostRange.startOffset + rel.startOffset, hostRange.startOffset + rel.endOffset)
findings += Finding(abs, extractMessageFromTypo(t) ?: "Spelling/Grammar")
}
}
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator: Grazie check failed: ${e.javaClass.simpleName}: ${e.message}")
}
}
log.info("LyngGrazieAnnotator.apply: used=${chosenEntry ?: "<none>"}, totalFindings=$totalReturned, painting=${findings.size}")
// IMPORTANT: Do NOT fallback to the tiny bundled vocabulary on modern IDEs.
// If Grazie/Natural Languages processing returned nothing, we simply exit here
// to avoid low‑quality results from the legacy dictionary.
if (findings.isEmpty()) return
for (f in findings) {
val ab = holder.newAnnotation(HighlightSeverity.INFORMATION, f.message).range(f.range)
applyTypoStyleIfRequested(file, ab)
ab.create()
}
}
private fun scheduleOneShotRestart(file: PsiFile, modStamp: Long) {
try {
val last = file.getUserData(RETRY_KEY)
if (last == modStamp) {
log.info("LyngGrazieAnnotator.restart: already retried for modStamp=$modStamp, skip")
return
}
file.putUserData(RETRY_KEY, modStamp)
ApplicationManager.getApplication().invokeLater({
try {
DaemonCodeAnalyzer.getInstance(file.project).restart(file)
log.info("LyngGrazieAnnotator.restart: daemon restarted for '${file.name}'")
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator.restart failed: ${e.javaClass.simpleName}: ${e.message}")
}
})
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator.scheduleOneShotRestart failed: ${e.javaClass.simpleName}: ${e.message}")
}
}
private fun isGrazieInstalled(): Boolean {
return PluginManagerCore.isPluginInstalled(com.intellij.openapi.extensions.PluginId.getId("com.intellij.grazie")) ||
PluginManagerCore.isPluginInstalled(com.intellij.openapi.extensions.PluginId.getId("tanvd.grazi"))
}
private fun rangeToTextContent(file: PsiFile, domain: TextDomain, range: TextRange): TextContent? {
// Build TextContent via reflection: prefer psiFragment(domain, element)
return try {
// Try to find an element that fully covers the target range
var element = file.findElementAt(range.startOffset) ?: return null
val start = range.startOffset
val end = range.endOffset
while (element.parent != null && (element.textRange.startOffset > start || element.textRange.endOffset < end)) {
element = element.parent
}
if (element.textRange.startOffset > start || element.textRange.endOffset < end) return null
// In many cases, the element may not span the whole range; use file + range via suitable factory
val methods = TextContent::class.java.methods.filter { it.name == "psiFragment" }
val byElementDomain = methods.firstOrNull { it.parameterCount == 2 && it.parameterTypes[0].name.endsWith("PsiElement") }
if (byElementDomain != null) {
@Suppress("UNCHECKED_CAST")
return (byElementDomain.invoke(null, element, domain) as? TextContent)?.let { tc ->
val relStart = start - element.textRange.startOffset
val relEnd = end - element.textRange.startOffset
if (relStart < 0 || relEnd > tc.length || relStart >= relEnd) return null
tc.subText(TextRange(relStart, relEnd))
}
}
val byDomainElement = methods.firstOrNull { it.parameterCount == 2 && it.parameterTypes[0].name.endsWith("TextDomain") }
if (byDomainElement != null) {
@Suppress("UNCHECKED_CAST")
return (byDomainElement.invoke(null, domain, element) as? TextContent)?.let { tc ->
val relStart = start - element.textRange.startOffset
val relEnd = end - element.textRange.startOffset
if (relStart < 0 || relEnd > tc.length || relStart >= relEnd) return null
tc.subText(TextRange(relStart, relEnd))
}
}
null
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator: rangeToTextContent failed: ${e.javaClass.simpleName}: ${e.message}")
null
}
}
private fun runGrazieChecksWithTracing(file: PsiFile, content: TextContent): Pair<Collection<Any>?, String?> {
// Try known entry points via reflection to avoid hard dependencies on Grazie internals
if (isGrammarCheckerKnownMissing()) return null to null
try {
// 1) Static GrammarChecker.check(TextContent)
val checkerCls = try {
Class.forName("com.intellij.grazie.grammar.GrammarChecker").also { grammarCheckerAvailable = true }
} catch (t: Throwable) {
grammarCheckerAvailable = false
markGrammarCheckerMissingOnce(log, "LyngGrazieAnnotator: GrammarChecker class not found: ${t.javaClass.simpleName}: ${t.message}")
null
}
if (checkerCls != null) {
// Diagnostic: list available 'check' methods once
runCatching {
val checks = checkerCls.methods.filter { it.name == "check" }
val sig = checks.joinToString { m ->
val params = m.parameterTypes.joinToString(prefix = "(", postfix = ")") { it.simpleName }
"${m.name}$params static=${java.lang.reflect.Modifier.isStatic(m.modifiers)}"
}
log.info("LyngGrazieAnnotator: GrammarChecker.check candidates: ${if (sig.isEmpty()) "<none>" else sig}")
}
checkerCls.methods.firstOrNull { it.name == "check" && it.parameterCount == 1 && it.parameterTypes[0].name.endsWith("TextContent") }?.let { m ->
@Suppress("UNCHECKED_CAST")
val res = m.invoke(null, content) as? Collection<Any>
return res to "GrammarChecker.check(TextContent) static"
}
// 2) GrammarChecker.getInstance().check(TextContent)
val getInstance = checkerCls.methods.firstOrNull { it.name == "getInstance" && it.parameterCount == 0 }
val inst = getInstance?.invoke(null)
if (inst != null) {
val m = checkerCls.methods.firstOrNull { it.name == "check" && it.parameterCount == 1 && it.parameterTypes[0].name.endsWith("TextContent") }
if (m != null) {
@Suppress("UNCHECKED_CAST")
val res = m.invoke(inst, content) as? Collection<Any>
return res to "GrammarChecker.getInstance().check(TextContent)"
}
}
// 3) GrammarChecker.getDefault().check(TextContent)
val getDefault = checkerCls.methods.firstOrNull { it.name == "getDefault" && it.parameterCount == 0 }
val def = getDefault?.invoke(null)
if (def != null) {
val m = checkerCls.methods.firstOrNull { it.name == "check" && it.parameterCount == 1 && it.parameterTypes[0].name.endsWith("TextContent") }
if (m != null) {
@Suppress("UNCHECKED_CAST")
val res = m.invoke(def, content) as? Collection<Any>
return res to "GrammarChecker.getDefault().check(TextContent)"
}
}
// 4) Service from project/application: GrammarChecker as a service
runCatching {
val app = com.intellij.openapi.application.ApplicationManager.getApplication()
val getService = app::class.java.methods.firstOrNull { it.name == "getService" && it.parameterCount == 1 }
val svc = getService?.invoke(app, checkerCls)
if (svc != null) {
val m = checkerCls.methods.firstOrNull { it.name == "check" && it.parameterCount == 1 && it.parameterTypes[0].name.endsWith("TextContent") }
if (m != null) {
@Suppress("UNCHECKED_CAST")
val res = m.invoke(svc, content) as? Collection<Any>
if (res != null) return res to "Application.getService(GrammarChecker).check(TextContent)"
}
}
}
runCatching {
val getService = file.project::class.java.methods.firstOrNull { it.name == "getService" && it.parameterCount == 1 }
val svc = getService?.invoke(file.project, checkerCls)
if (svc != null) {
val m = checkerCls.methods.firstOrNull { it.name == "check" && it.parameterCount == 1 && it.parameterTypes[0].name.endsWith("TextContent") }
if (m != null) {
@Suppress("UNCHECKED_CAST")
val res = m.invoke(svc, content) as? Collection<Any>
if (res != null) return res to "Project.getService(GrammarChecker).check(TextContent)"
}
}
}
}
// 5) Fallback: search any public method named check that accepts TextContent in any Grazie class (static)
val candidateClasses = listOf(
"com.intellij.grazie.grammar.GrammarChecker",
"com.intellij.grazie.grammar.GrammarRunner",
"com.intellij.grazie.grammar.Grammar" // historical names
)
for (cn in candidateClasses) {
val cls = try { Class.forName(cn) } catch (_: Throwable) { continue }
val m = cls.methods.firstOrNull { it.name == "check" && it.parameterTypes.any { p -> p.name.endsWith("TextContent") } }
if (m != null) {
val args = arrayOfNulls<Any>(m.parameterCount)
// place content to the first TextContent parameter; others left null (common defaults)
for (i in 0 until m.parameterCount) if (m.parameterTypes[i].name.endsWith("TextContent")) { args[i] = content; break }
@Suppress("UNCHECKED_CAST")
val res = m.invoke(null, *args) as? Collection<Any>
if (res != null) return res to "$cn.${m.name}(TextContent)"
}
}
// 6) Kotlin top-level function: GrammarCheckerKt.check(TextContent)
runCatching {
val kt = Class.forName("com.intellij.grazie.grammar.GrammarCheckerKt")
val m = kt.methods.firstOrNull { it.name == "check" && it.parameterTypes.any { p -> p.name.endsWith("TextContent") } }
if (m != null) {
val args = arrayOfNulls<Any>(m.parameterCount)
for (i in 0 until m.parameterCount) if (m.parameterTypes[i].name.endsWith("TextContent")) { args[i] = content; break }
@Suppress("UNCHECKED_CAST")
val res = m.invoke(null, *args) as? Collection<Any>
if (res != null) return res to "GrammarCheckerKt.check(TextContent)"
}
}
} catch (e: Throwable) {
log.info("LyngGrazieAnnotator: runGrazieChecks reflection failed: ${e.javaClass.simpleName}: ${e.message}")
}
return null to null
}
private fun extractRangeFromTypo(typo: Any): TextRange? {
// Try to get a relative range from returned Grazie issue/typo via common accessors
return try {
// Common getters
val m1 = typo.javaClass.methods.firstOrNull { it.name == "getRange" && it.parameterCount == 0 }
val r1 = if (m1 != null) m1.invoke(typo) else null
when (r1) {
is TextRange -> return r1
is IntRange -> return TextRange(r1.first, r1.last + 1)
}
val m2 = typo.javaClass.methods.firstOrNull { it.name == "getHighlightRange" && it.parameterCount == 0 }
val r2 = if (m2 != null) m2.invoke(typo) else null
when (r2) {
is TextRange -> return r2
is IntRange -> return TextRange(r2.first, r2.last + 1)
}
// Separate from/to ints
val fromM = typo.javaClass.methods.firstOrNull { it.name == "getFrom" && it.parameterCount == 0 && it.returnType == Int::class.javaPrimitiveType }
val toM = typo.javaClass.methods.firstOrNull { it.name == "getTo" && it.parameterCount == 0 && it.returnType == Int::class.javaPrimitiveType }
if (fromM != null && toM != null) {
val s = (fromM.invoke(typo) as? Int) ?: return null
val e = (toM.invoke(typo) as? Int) ?: return null
if (e > s) return TextRange(s, e)
}
null
} catch (_: Throwable) { null }
}
private fun extractMessageFromTypo(typo: Any): String? {
return try {
val m = typo.javaClass.methods.firstOrNull { it.name == "getMessage" && it.parameterCount == 0 }
(m?.invoke(typo) as? String)
} catch (_: Throwable) { null }
}
// Fallback that uses legacy SpellCheckerManager (if present) via reflection to validate words in fragments.
// Returns number of warnings painted.
private fun fallbackWithLegacySpellcheckerIfAvailable(
file: PsiFile,
fragments: List<Pair<TextContent, TextRange>>,
holder: AnnotationHolder
): Int {
return try {
val mgrCls = Class.forName("com.intellij.spellchecker.SpellCheckerManager")
val getInstance = mgrCls.methods.firstOrNull { it.name == "getInstance" && it.parameterCount == 1 }
val isCorrect = mgrCls.methods.firstOrNull { it.name == "isCorrect" && it.parameterCount == 1 && it.parameterTypes[0] == String::class.java }
if (getInstance == null || isCorrect == null) {
// No legacy spellchecker API available — fall back to naive painter
return naiveFallbackPaint(file, fragments, holder)
}
val mgr = getInstance.invoke(null, file.project)
if (mgr == null) {
// Legacy manager not present for this project — use naive fallback
return naiveFallbackPaint(file, fragments, holder)
}
var painted = 0
val docText = file.viewProvider.document?.text ?: return 0
val tokenRegex = Regex("[A-Za-z][A-Za-z0-9_']{2,}")
for ((content, hostRange) in fragments) {
val text = try { docText.substring(hostRange.startOffset, hostRange.endOffset) } catch (_: Throwable) { null } ?: continue
var seen = 0
var flagged = 0
for (m in tokenRegex.findAll(text)) {
val token = m.value
if ('%' in token) continue // skip printf fragments defensively
// Split snake_case and camelCase within the token
val parts = splitIdentifier(token)
for (part in parts) {
if (part.length <= 2) continue
if (isAllowedWord(part)) continue
// Quick allowlist for very common words to reduce noise if dictionaries differ
val ok = try { isCorrect.invoke(mgr, part) as? Boolean } catch (_: Throwable) { null }
if (ok == false) {
// Map part back to original token occurrence within this hostRange
val localStart = m.range.first + token.indexOf(part)
val localEnd = localStart + part.length
val abs = TextRange(hostRange.startOffset + localStart, hostRange.startOffset + localEnd)
paintTypoAnnotation(file, holder, abs, part)
painted++
flagged++
}
seen++
}
}
log.info("LyngGrazieAnnotator.fallback: fragment words=$seen, flagged=$flagged")
}
painted
} catch (_: Throwable) {
// If legacy manager is not available, fall back to a very naive heuristic (no external deps)
return naiveFallbackPaint(file, fragments, holder)
}
}
private fun naiveFallbackPaint(
file: PsiFile,
fragments: List<Pair<TextContent, TextRange>>,
holder: AnnotationHolder
): Int {
var painted = 0
val docText = file.viewProvider.document?.text
val tokenRegex = Regex("[A-Za-z][A-Za-z0-9_']{2,}")
val baseWords = setOf(
// small, common vocabulary to catch near-miss typos in typical code/comments
"comment","comments","error","errors","found","file","not","word","words","count","value","name","class","function","string"
)
for ((content, hostRange) in fragments) {
val text: String? = docText?.let { dt ->
try { dt.substring(hostRange.startOffset, hostRange.endOffset) } catch (_: Throwable) { null }
}
if (text.isNullOrBlank()) continue
var seen = 0
var flagged = 0
for (m in tokenRegex.findAll(text)) {
val token = m.value
if ('%' in token) continue
val parts = splitIdentifier(token)
for (part in parts) {
seen++
val lower = part.lowercase()
if (lower.length <= 2 || isAllowedWord(part)) continue
// Heuristic: no vowels OR 3 repeated chars OR ends with unlikely double consonants
val noVowel = lower.none { it in "aeiouy" }
val triple = Regex("(.)\\1\\1").containsMatchIn(lower)
val dblCons = Regex("[bcdfghjklmnpqrstvwxyz]{2}$").containsMatchIn(lower)
var looksWrong = noVowel || triple || dblCons
// Additional: low vowel ratio for length>=4
if (!looksWrong && lower.length >= 4) {
val vowels = lower.count { it in "aeiouy" }
val ratio = if (lower.isNotEmpty()) vowels.toDouble() / lower.length else 1.0
if (ratio < 0.25) looksWrong = true
}
// Additional: near-miss to a small base vocabulary (edit distance 1, or 2 for words >=6)
if (!looksWrong) {
for (bw in baseWords) {
val d = editDistance(lower, bw)
if (d == 1 || (d == 2 && lower.length >= 6)) { looksWrong = true; break }
}
}
if (looksWrong) {
val localStart = m.range.first + token.indexOf(part)
val localEnd = localStart + part.length
val abs = TextRange(hostRange.startOffset + localStart, hostRange.startOffset + localEnd)
paintTypoAnnotation(file, holder, abs, part)
painted++
flagged++
}
}
}
log.info("LyngGrazieAnnotator.fallback(naive): fragment words=$seen, flagged=$flagged")
}
return painted
}
private fun paintTypoAnnotation(file: PsiFile, holder: AnnotationHolder, range: TextRange, word: String) {
val settings = LyngFormatterSettings.getInstance(file.project)
val ab = holder.newAnnotation(HighlightSeverity.INFORMATION, "Possible typo")
.range(range)
applyTypoStyleIfRequested(file, ab)
if (settings.offerLyngTypoQuickFixes) {
// Offer lightweight fixes; for 243 provide Add-to-dictionary always
ab.withFix(net.sergeych.lyng.idea.grazie.AddToLyngDictionaryFix(word))
// Offer "Replace with…" candidates (top 7)
val cands = suggestReplacements(file, word).take(7)
for (c in cands) {
ab.withFix(net.sergeych.lyng.idea.grazie.ReplaceWordFix(range, word, c))
}
}
ab.create()
}
private fun applyTypoStyleIfRequested(file: PsiFile, ab: com.intellij.lang.annotation.AnnotationBuilder) {
val settings = LyngFormatterSettings.getInstance(file.project)
if (!settings.showTyposWithGreenUnderline) return
// Use the standard TYPO text attributes key used by the platform
val TYPO: TextAttributesKey = TextAttributesKey.createTextAttributesKey("TYPO")
try {
ab.textAttributes(TYPO)
} catch (_: Throwable) {
// some IDEs may not allow setting attributes on INFORMATION; ignore gracefully
}
}
private fun suggestReplacements(file: PsiFile, word: String): List<String> {
val lower = word.lowercase()
val fromProject = collectProjectWords(file)
val fromTech = TechDictionary.allWords()
val fromEnglish = EnglishDictionary.allWords()
// Merge with priority: project (p=0), tech (p=1), english (p=2)
val all = LinkedHashSet<String>()
all.addAll(fromProject)
all.addAll(fromTech)
all.addAll(fromEnglish)
data class Cand(val w: String, val d: Int, val p: Int)
val cands = ArrayList<Cand>(32)
for (w in all) {
if (w == lower) continue
if (kotlin.math.abs(w.length - lower.length) > 2) continue
val d = editDistance(lower, w)
val p = when {
w in fromProject -> 0
w in fromTech -> 1
else -> 2
}
cands += Cand(w, d, p)
}
cands.sortWith(compareBy<Cand> { it.d }.thenBy { it.p }.thenBy { it.w })
// Return a larger pool so callers can choose desired display count
return cands.take(16).map { it.w }
}
private fun collectProjectWords(file: PsiFile): Set<String> {
// Simple approach: use current file text; can be extended to project scanning later
val text = file.viewProvider.document?.text ?: return emptySet()
val out = LinkedHashSet<String>()
val tokenRegex = Regex("[A-Za-z][A-Za-z0-9_']{2,}")
for (m in tokenRegex.findAll(text)) {
val parts = splitIdentifier(m.value)
parts.forEach { out += it.lowercase() }
}
// Include learned words
val settings = LyngFormatterSettings.getInstance(file.project)
out.addAll(settings.learnedWords.map { it.lowercase() })
return out
}
private fun splitIdentifier(token: String): List<String> {
// Split on underscores and camelCase boundaries
val unders = token.split('_').filter { it.isNotBlank() }
val out = mutableListOf<String>()
val camelBoundary = Regex("(?<=[a-z])(?=[A-Z])")
for (u in unders) out += u.split(camelBoundary).filter { it.isNotBlank() }
return out
}
private fun isAllowedWord(w: String): Boolean {
val s = w.lowercase()
return s in setOf(
// common code words / language keywords to avoid noise
"val","var","fun","class","enum","type","import","package","return","if","else","when","while","for","try","catch","finally","true","false","null",
// very common English words
"the","and","or","not","with","from","into","this","that","file","found","count","name","value","object"
)
}
private fun editDistance(a: String, b: String): Int {
if (a == b) return 0
if (a.isEmpty()) return b.length
if (b.isEmpty()) return a.length
val dp = IntArray(b.length + 1) { it }
for (i in 1..a.length) {
var prev = dp[0]
dp[0] = i
for (j in 1..b.length) {
val temp = dp[j]
dp[j] = minOf(
dp[j] + 1, // deletion
dp[j - 1] + 1, // insertion
prev + if (a[i - 1] == b[j - 1]) 0 else 1 // substitution
)
prev = temp
}
}
return dp[b.length]
}
}
@@ -0,0 +1,139 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.grazie.grammar.strategy.GrammarCheckingStrategy
import com.intellij.grazie.grammar.strategy.GrammarCheckingStrategy.TextDomain
import com.intellij.ide.plugins.PluginManagerCore
import com.intellij.openapi.diagnostic.Logger
import com.intellij.openapi.extensions.PluginId
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiElement
import net.sergeych.lyng.idea.highlight.LyngTokenTypes
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import net.sergeych.lyng.idea.spell.LyngSpellIndex
/**
* Grazie/Natural Languages strategy for Lyng.
*
* - Comments: checked as natural language (TextDomain.COMMENTS)
* - String literals: optionally checked (setting), skipping printf-like specifiers via stealth ranges (TextDomain.LITERALS)
* - Identifiers (non-keywords): checked under TextDomain.CODE so "Process code" controls apply
* - Keywords: skipped
*/
class LyngGrazieStrategy : GrammarCheckingStrategy {
private val log = Logger.getInstance(LyngGrazieStrategy::class.java)
@Volatile private var loggedOnce = false
@Volatile private var loggedFirstMatch = false
private val seenTypes: MutableSet<String> = java.util.Collections.synchronizedSet(mutableSetOf())
private fun legacySpellcheckerInstalled(): Boolean =
PluginManagerCore.isPluginInstalled(PluginId.getId("com.intellij.spellchecker"))
// Regex for printf-style specifiers: %[flags][width][.precision][length]type
private val spec = Regex("%(?:[-+ #0]*(?:\\d+)?(?:\\.\\d+)?[a-zA-Z%])")
override fun isMyContextRoot(element: PsiElement): Boolean {
val type = element.node?.elementType
val settings = LyngFormatterSettings.getInstance(element.project)
val legacyPresent = legacySpellcheckerInstalled()
if (type != null && seenTypes.size < 10) {
val name = type.toString()
if (seenTypes.add(name)) {
log.info("LyngGrazieStrategy: saw PSI type=$name")
}
}
if (!loggedOnce) {
loggedOnce = true
log.info("LyngGrazieStrategy activated: legacyPresent=$legacyPresent, preferGrazieForCommentsAndLiterals=${settings.preferGrazieForCommentsAndLiterals}, spellCheckStringLiterals=${settings.spellCheckStringLiterals}, grazieChecksIdentifiers=${settings.grazieChecksIdentifiers}")
}
val file = element.containingFile ?: return false
val index = LyngSpellIndex.getUpToDate(file) ?: return false // Suspend until ready
// To ensure Grazie asks TextExtractor for all leafs, accept any Lyng element once index is ready.
// The extractor will decide per-range/domain what to actually provide.
if (!loggedFirstMatch) {
loggedFirstMatch = true
log.info("LyngGrazieStrategy: enabling Grazie on all Lyng elements (index ready)")
}
return true
}
override fun getContextRootTextDomain(root: PsiElement): TextDomain {
val type = root.node?.elementType
val settings = LyngFormatterSettings.getInstance(root.project)
val file = root.containingFile
val index = if (file != null) LyngSpellIndex.getUpToDate(file) else null
val r = root.textRange
fun overlaps(list: List<TextRange>): Boolean = r != null && list.any { it.intersects(r) }
return when (type) {
LyngTokenTypes.LINE_COMMENT, LyngTokenTypes.BLOCK_COMMENT -> TextDomain.COMMENTS
LyngTokenTypes.STRING -> if (settings.grazieTreatLiteralsAsComments) TextDomain.COMMENTS else TextDomain.LITERALS
LyngTokenTypes.IDENTIFIER -> {
// For Grazie-only reliability in 243, route identifiers via COMMENTS when configured
if (settings.grazieTreatIdentifiersAsComments && index != null && r != null && overlaps(index.identifiers))
TextDomain.COMMENTS
else TextDomain.PLAIN_TEXT
}
else -> TextDomain.PLAIN_TEXT
}
}
// Note: do not override getLanguageSupport to keep compatibility with 243 API
override fun getStealthyRanges(root: PsiElement, text: CharSequence): java.util.LinkedHashSet<IntRange> {
val result = LinkedHashSet<IntRange>()
val type = root.node?.elementType
if (type == LyngTokenTypes.STRING) {
if (!shouldCheckLiterals(root)) {
// Hide the entire string when literals checking is disabled by settings
result += (0 until text.length)
return result
}
// Hide printf-like specifiers in strings
val (start, end) = stripQuotesBounds(text)
if (end > start) {
val content = text.subSequence(start, end)
for (m in spec.findAll(content)) {
val ms = start + m.range.first
val me = start + m.range.last
result += (ms..me)
}
if (result.isNotEmpty()) {
log.debug("LyngGrazieStrategy: hidden ${result.size} printf specifier ranges in string literal")
}
}
}
return result
}
override fun isEnabledByDefault(): Boolean = true
private fun shouldCheckLiterals(root: PsiElement): Boolean =
LyngFormatterSettings.getInstance(root.project).spellCheckStringLiterals
private fun stripQuotesBounds(text: CharSequence): Pair<Int, Int> {
if (text.length < 2) return 0 to text.length
val first = text.first()
val last = text.last()
return if ((first == '"' && last == '"') || (first == '\'' && last == '\''))
1 to (text.length - 1) else (0 to text.length)
}
}
@@ -0,0 +1,104 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.grazie.text.TextContent
import com.intellij.grazie.text.TextContent.TextDomain
import com.intellij.grazie.text.TextExtractor
import com.intellij.openapi.diagnostic.Logger
import com.intellij.psi.PsiElement
import net.sergeych.lyng.idea.highlight.LyngTokenTypes
import net.sergeych.lyng.idea.settings.LyngFormatterSettings
import net.sergeych.lyng.idea.spell.LyngSpellIndex
/**
* Provides Grazie with extractable text for Lyng PSI elements.
* We return text for identifiers, comments, and (optionally) string literals.
* printf-like specifiers are filtered by the Grammar strategy via stealth ranges.
*/
class LyngTextExtractor : TextExtractor() {
private val log = Logger.getInstance(LyngTextExtractor::class.java)
@Volatile private var loggedOnce = false
private val seen: MutableSet<String> = java.util.Collections.synchronizedSet(mutableSetOf())
override fun buildTextContent(element: PsiElement, allowedDomains: Set<TextDomain>): TextContent? {
val type = element.node?.elementType ?: return null
if (!loggedOnce) {
loggedOnce = true
log.info("LyngTextExtractor active; allowedDomains=${allowedDomains.joinToString()}")
}
val settings = LyngFormatterSettings.getInstance(element.project)
val file = element.containingFile
val index = if (file != null) LyngSpellIndex.getUpToDate(file) else null
val r = element.textRange
fun overlaps(list: List<com.intellij.openapi.util.TextRange>): Boolean = r != null && list.any { it.intersects(r) }
// Decide target domain by intersection with our MiniAst-driven index; prefer comments > strings > identifiers
var domain: TextDomain? = null
if (index != null && r != null) {
if (overlaps(index.comments)) domain = TextDomain.COMMENTS
else if (overlaps(index.strings) && settings.spellCheckStringLiterals) domain = TextDomain.LITERALS
else if (overlaps(index.identifiers)) domain = if (settings.grazieTreatIdentifiersAsComments) TextDomain.COMMENTS else TextDomain.DOCUMENTATION
} else {
// Fallback to token type if index is not ready (rare timing), mostly for comments
domain = when (type) {
LyngTokenTypes.LINE_COMMENT, LyngTokenTypes.BLOCK_COMMENT -> TextDomain.COMMENTS
else -> null
}
}
if (domain == null) return null
// If literals aren't requested but fallback is enabled, route strings as COMMENTS
if (domain == TextDomain.LITERALS && !allowedDomains.contains(TextDomain.LITERALS) && settings.grazieTreatLiteralsAsComments) {
domain = TextDomain.COMMENTS
}
if (!allowedDomains.contains(domain)) {
if (seen.add("deny-${domain.name}")) {
log.info("LyngTextExtractor: domain ${domain.name} not in allowedDomains; skipping")
}
return null
}
return try {
// Try common factory names across versions
val methods = TextContent::class.java.methods.filter { it.name == "psiFragment" }
val built: TextContent? = when {
// Try psiFragment(PsiElement, TextDomain)
methods.any { it.parameterCount == 2 && it.parameterTypes[0].name.contains("PsiElement") } -> {
val m = methods.first { it.parameterCount == 2 && it.parameterTypes[0].name.contains("PsiElement") }
@Suppress("UNCHECKED_CAST")
(m.invoke(null, element, domain) as? TextContent)?.also {
if (seen.add("ok-${domain.name}")) log.info("LyngTextExtractor: provided ${domain.name} for ${type} via psiFragment(element, domain)")
}
}
// Try psiFragment(TextDomain, PsiElement)
methods.any { it.parameterCount == 2 && it.parameterTypes[0].name.endsWith("TextDomain") } -> {
val m = methods.first { it.parameterCount == 2 && it.parameterTypes[0].name.endsWith("TextDomain") }
@Suppress("UNCHECKED_CAST")
(m.invoke(null, domain, element) as? TextContent)?.also {
if (seen.add("ok-${domain.name}")) log.info("LyngTextExtractor: provided ${domain.name} for ${type} via psiFragment(domain, element)")
}
}
else -> null
}
built
} catch (e: Throwable) {
log.info("LyngTextExtractor: failed to build TextContent: ${e.javaClass.simpleName}: ${e.message}")
null
}
}
}
@@ -0,0 +1,86 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.codeInsight.daemon.DaemonCodeAnalyzer
import com.intellij.codeInsight.intention.IntentionAction
import com.intellij.openapi.command.WriteCommandAction
import com.intellij.openapi.editor.CaretModel
import com.intellij.openapi.editor.Document
import com.intellij.openapi.editor.Editor
import com.intellij.openapi.project.Project
import com.intellij.openapi.util.TextRange
import com.intellij.psi.PsiFile
/**
* Lightweight quick-fix to replace a misspelled word (subrange) with a suggested alternative.
* Works without the legacy Spell Checker. The replacement is applied directly to the file text.
*/
class ReplaceWordFix(
private val range: TextRange,
private val original: String,
private val replacementRaw: String
) : IntentionAction {
override fun getText(): String = "Replace '$original' with '$replacementRaw'"
override fun getFamilyName(): String = "Lyng Spelling"
override fun isAvailable(project: Project, editor: Editor?, file: PsiFile?): Boolean =
editor != null && file != null && range.startOffset in 0..range.endOffset
override fun startInWriteAction(): Boolean = true
override fun invoke(project: Project, editor: Editor?, file: PsiFile?) {
if (editor == null) return
val doc: Document = editor.document
val safeRange = range.constrainTo(doc)
val current = doc.getText(safeRange)
// Preserve basic case style based on the original token
val replacement = adaptCaseStyle(current, replacementRaw)
WriteCommandAction.runWriteCommandAction(project, "Replace word", null, Runnable {
doc.replaceString(safeRange.startOffset, safeRange.endOffset, replacement)
}, file)
// Move caret to end of replacement for convenience
try {
val caret: CaretModel = editor.caretModel
caret.moveToOffset(safeRange.startOffset + replacement.length)
} catch (_: Throwable) {}
// Restart daemon to refresh highlights
if (file != null) DaemonCodeAnalyzer.getInstance(project).restart(file)
}
private fun TextRange.constrainTo(doc: Document): TextRange {
val start = startOffset.coerceIn(0, doc.textLength)
val end = endOffset.coerceIn(start, doc.textLength)
return TextRange(start, end)
}
private fun adaptCaseStyle(sample: String, suggestion: String): String {
if (suggestion.isEmpty()) return suggestion
return when {
sample.all { it.isUpperCase() } -> suggestion.uppercase()
// PascalCase / Capitalized single word
sample.firstOrNull()?.isUpperCase() == true && sample.drop(1).any { it.isLowerCase() } ->
suggestion.replaceFirstChar { if (it.isLowerCase()) it.titlecase() else it.toString() }
// snake_case -> lower
sample.contains('_') -> suggestion.lowercase()
// camelCase -> lower first
sample.firstOrNull()?.isLowerCase() == true && sample.any { it.isUpperCase() } ->
suggestion.replaceFirstChar { it.lowercase() }
else -> suggestion
}
}
}
@@ -0,0 +1,77 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.grazie
import com.intellij.openapi.diagnostic.Logger
import java.io.BufferedReader
import java.io.InputStreamReader
import java.util.zip.GZIPInputStream
/**
* Lightweight technical/Lyng vocabulary dictionary.
* Loaded from classpath resources; supports .txt and .txt.gz. Merged with EnglishDictionary.
*/
object TechDictionary {
private val log = Logger.getInstance(TechDictionary::class.java)
@Volatile private var loaded = false
@Volatile private var words: Set<String> = emptySet()
private fun ensureLoaded() {
if (loaded) return
synchronized(this) {
if (loaded) return
val candidates = listOf(
"/dictionaries/tech-lyng.txt.gz",
"/dictionaries/tech-lyng.txt"
)
val merged = HashSet<String>(8_000)
for (res in candidates) {
try {
val stream = javaClass.getResourceAsStream(res) ?: continue
val reader = if (res.endsWith(".gz"))
BufferedReader(InputStreamReader(GZIPInputStream(stream)))
else
BufferedReader(InputStreamReader(stream))
var n = 0
reader.useLines { seq -> seq.forEach { line ->
val w = line.trim()
if (w.isNotEmpty() && !w.startsWith("#")) { merged += w.lowercase(); n++ }
} }
log.info("TechDictionary: loaded $n words from $res (total=${merged.size})")
} catch (t: Throwable) {
log.info("TechDictionary: failed to load $res: ${t.javaClass.simpleName}: ${t.message}")
}
}
if (merged.isEmpty()) {
merged += setOf(
// minimal Lyng/tech seeding to avoid empty dictionary
"lyng","miniast","binder","printf","specifier","specifiers","regex","token","tokens",
"identifier","identifiers","keyword","keywords","comment","comments","string","strings",
"literal","literals","formatting","formatter","grazie","typo","typos","dictionary","dictionaries"
)
log.info("TechDictionary: using minimal built-in set (${merged.size})")
}
words = merged
loaded = true
}
}
fun allWords(): Set<String> {
ensureLoaded()
return words
}
}
@@ -0,0 +1,72 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.highlight
import com.intellij.openapi.editor.colors.TextAttributesKey
import com.intellij.openapi.fileTypes.SyntaxHighlighter
import com.intellij.openapi.options.colors.AttributesDescriptor
import com.intellij.openapi.options.colors.ColorDescriptor
import com.intellij.openapi.options.colors.ColorSettingsPage
import javax.swing.Icon
class LyngColorSettingsPage : ColorSettingsPage {
override fun getDisplayName(): String = "Lyng"
override fun getIcon(): Icon? = null
override fun getHighlighter(): SyntaxHighlighter = LyngSyntaxHighlighter()
override fun getDemoText(): String = """
// Lyng demo
import lyng.stdlib as std
class Sample {
fun greet(name: String): String {
val message = "Hello, " + name
return message
}
}
var counter = 0
counter = counter + 1
""".trimIndent()
override fun getAdditionalHighlightingTagToDescriptorMap(): MutableMap<String, TextAttributesKey>? = null
override fun getAttributeDescriptors(): Array<AttributesDescriptor> = arrayOf(
AttributesDescriptor("Keyword", LyngHighlighterColors.KEYWORD),
AttributesDescriptor("String", LyngHighlighterColors.STRING),
AttributesDescriptor("Number", LyngHighlighterColors.NUMBER),
AttributesDescriptor("Line comment", LyngHighlighterColors.LINE_COMMENT),
AttributesDescriptor("Block comment", LyngHighlighterColors.BLOCK_COMMENT),
AttributesDescriptor("Identifier", LyngHighlighterColors.IDENTIFIER),
AttributesDescriptor("Punctuation", LyngHighlighterColors.PUNCT),
// Semantic
AttributesDescriptor("Annotation (semantic)", LyngHighlighterColors.ANNOTATION),
AttributesDescriptor("Variable (semantic)", LyngHighlighterColors.VARIABLE),
AttributesDescriptor("Value (semantic)", LyngHighlighterColors.VALUE),
AttributesDescriptor("Function (semantic)", LyngHighlighterColors.FUNCTION),
AttributesDescriptor("Function declaration (semantic)", LyngHighlighterColors.FUNCTION_DECLARATION),
AttributesDescriptor("Type (semantic)", LyngHighlighterColors.TYPE),
AttributesDescriptor("Namespace (semantic)", LyngHighlighterColors.NAMESPACE),
AttributesDescriptor("Parameter (semantic)", LyngHighlighterColors.PARAMETER),
AttributesDescriptor("Enum constant (semantic)", LyngHighlighterColors.ENUM_CONSTANT),
)
override fun getColorDescriptors(): Array<ColorDescriptor> = ColorDescriptor.EMPTY_ARRAY
}
@@ -0,0 +1,85 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* Text attribute keys for Lyng token and semantic highlighting
*/
package net.sergeych.lyng.idea.highlight
import com.intellij.openapi.editor.DefaultLanguageHighlighterColors
import com.intellij.openapi.editor.colors.TextAttributesKey
object LyngHighlighterColors {
val KEYWORD: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_KEYWORD", DefaultLanguageHighlighterColors.KEYWORD
)
val STRING: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_STRING", DefaultLanguageHighlighterColors.STRING
)
val NUMBER: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_NUMBER", DefaultLanguageHighlighterColors.NUMBER
)
val LINE_COMMENT: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_LINE_COMMENT", DefaultLanguageHighlighterColors.LINE_COMMENT
)
val BLOCK_COMMENT: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_BLOCK_COMMENT", DefaultLanguageHighlighterColors.BLOCK_COMMENT
)
val IDENTIFIER: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_IDENTIFIER", DefaultLanguageHighlighterColors.IDENTIFIER
)
val PUNCT: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_PUNCT", DefaultLanguageHighlighterColors.DOT
)
// Semantic layer keys
val VARIABLE: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
// Use a distinctive default to ensure visibility across common themes.
// Users can still customize it separately from VALUE.
"LYNG_VARIABLE", DefaultLanguageHighlighterColors.INSTANCE_FIELD
)
val VALUE: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_VALUE", DefaultLanguageHighlighterColors.INSTANCE_FIELD
)
val FUNCTION: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
// Primary approach: make function calls as visible as declarations by default
// (users can still customize separately in the color scheme UI).
"LYNG_FUNCTION", DefaultLanguageHighlighterColors.FUNCTION_DECLARATION
)
val FUNCTION_DECLARATION: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_FUNCTION_DECLARATION", DefaultLanguageHighlighterColors.FUNCTION_DECLARATION
)
val TYPE: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_TYPE", DefaultLanguageHighlighterColors.CLASS_REFERENCE
)
val NAMESPACE: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_NAMESPACE", DefaultLanguageHighlighterColors.PREDEFINED_SYMBOL
)
val PARAMETER: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_PARAMETER", DefaultLanguageHighlighterColors.PARAMETER
)
// Annotations (@Something) — use Kotlin/Java metadata default color
val ANNOTATION: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_ANNOTATION", DefaultLanguageHighlighterColors.METADATA
)
// Enum constant (declaration or usage)
val ENUM_CONSTANT: TextAttributesKey = TextAttributesKey.createTextAttributesKey(
"LYNG_ENUM_CONSTANT", DefaultLanguageHighlighterColors.STATIC_FIELD
)
}
@@ -0,0 +1,162 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
/*
* Minimal hand-written lexer for Lyng token highlighting
*/
package net.sergeych.lyng.idea.highlight
import com.intellij.lexer.LexerBase
import com.intellij.psi.tree.IElementType
class LyngLexer : LexerBase() {
private var buffer: CharSequence = ""
private var startOffset: Int = 0
private var endOffset: Int = 0
private var myTokenStart: Int = 0
private var myTokenEnd: Int = 0
private var myTokenType: IElementType? = null
private val keywords = setOf(
"fun", "val", "var", "class", "type", "import", "as",
"if", "else", "for", "while", "return", "true", "false", "null",
"when", "in", "is", "break", "continue", "try", "catch", "finally"
)
override fun start(buffer: CharSequence, startOffset: Int, endOffset: Int, initialState: Int) {
this.buffer = buffer
this.startOffset = startOffset
this.endOffset = endOffset
this.myTokenStart = startOffset
this.myTokenEnd = startOffset
this.myTokenType = null
advance()
}
override fun getState(): Int = 0
override fun getTokenType(): IElementType? = myTokenType
override fun getTokenStart(): Int = myTokenStart
override fun getTokenEnd(): Int = myTokenEnd
override fun getBufferSequence(): CharSequence = buffer
override fun getBufferEnd(): Int = endOffset
override fun advance() {
if (myTokenEnd >= endOffset) {
myTokenType = null
return
}
var i = if (myTokenEnd == 0) startOffset else myTokenEnd
// Skip nothing; set start
myTokenStart = i
if (i >= endOffset) { myTokenType = null; return }
val ch = buffer[i]
// Whitespace
if (ch.isWhitespace()) {
i++
while (i < endOffset && buffer[i].isWhitespace()) i++
myTokenEnd = i
myTokenType = LyngTokenTypes.WHITESPACE
return
}
// Line comment //...
if (ch == '/' && i + 1 < endOffset && buffer[i + 1] == '/') {
i += 2
while (i < endOffset && buffer[i] != '\n' && buffer[i] != '\r') i++
myTokenEnd = i
myTokenType = LyngTokenTypes.LINE_COMMENT
return
}
// Block comment /* ... */
if (ch == '/' && i + 1 < endOffset && buffer[i + 1] == '*') {
i += 2
while (i + 1 < endOffset && !(buffer[i] == '*' && buffer[i + 1] == '/')) i++
if (i + 1 < endOffset) i += 2 // consume */
myTokenEnd = i
myTokenType = LyngTokenTypes.BLOCK_COMMENT
return
}
// String "..." with simple escape handling
if (ch == '"') {
i++
while (i < endOffset) {
val c = buffer[i]
if (c == '\\') { // escape
i += 2
continue
}
if (c == '"') { i++; break }
i++
}
myTokenEnd = i
myTokenType = LyngTokenTypes.STRING
return
}
// Number
if (ch.isDigit()) {
i++
var hasDot = false
while (i < endOffset) {
val c = buffer[i]
if (c.isDigit()) { i++; continue }
if (c == '.' && !hasDot) { hasDot = true; i++; continue }
break
}
myTokenEnd = i
myTokenType = LyngTokenTypes.NUMBER
return
}
// Identifier / keyword
if (ch.isIdentifierStart()) {
i++
while (i < endOffset && buffer[i].isIdentifierPart()) i++
myTokenEnd = i
val text = buffer.subSequence(myTokenStart, myTokenEnd).toString()
myTokenType = if (text in keywords) LyngTokenTypes.KEYWORD else LyngTokenTypes.IDENTIFIER
return
}
// Punctuation
if (isPunct(ch)) {
i++
myTokenEnd = i
myTokenType = LyngTokenTypes.PUNCT
return
}
// Fallback bad char
myTokenEnd = i + 1
myTokenType = LyngTokenTypes.BAD_CHAR
}
private fun Char.isWhitespace(): Boolean = this == ' ' || this == '\t' || this == '\n' || this == '\r' || this == '\u000C'
private fun Char.isDigit(): Boolean = this in '0'..'9'
private fun Char.isIdentifierStart(): Boolean = this == '_' || this.isLetter()
private fun Char.isIdentifierPart(): Boolean = this.isIdentifierStart() || this.isDigit()
private fun isPunct(c: Char): Boolean = c in setOf('(', ')', '{', '}', '[', ']', '.', ',', ';', ':', '+', '-', '*', '/', '%', '=', '<', '>', '!', '?', '&', '|', '^', '~')
}
@@ -0,0 +1,40 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.highlight
import com.intellij.lexer.Lexer
import com.intellij.openapi.editor.colors.TextAttributesKey
import com.intellij.openapi.fileTypes.SyntaxHighlighter
import com.intellij.psi.tree.IElementType
class LyngSyntaxHighlighter : SyntaxHighlighter {
override fun getHighlightingLexer(): Lexer = LyngLexer()
override fun getTokenHighlights(tokenType: IElementType): Array<TextAttributesKey> = when (tokenType) {
LyngTokenTypes.KEYWORD -> pack(LyngHighlighterColors.KEYWORD)
LyngTokenTypes.STRING -> pack(LyngHighlighterColors.STRING)
LyngTokenTypes.NUMBER -> pack(LyngHighlighterColors.NUMBER)
LyngTokenTypes.LINE_COMMENT -> pack(LyngHighlighterColors.LINE_COMMENT)
LyngTokenTypes.BLOCK_COMMENT -> pack(LyngHighlighterColors.BLOCK_COMMENT)
LyngTokenTypes.PUNCT -> pack(LyngHighlighterColors.PUNCT)
LyngTokenTypes.IDENTIFIER -> pack(LyngHighlighterColors.IDENTIFIER)
else -> emptyArray()
}
private fun pack(vararg keys: TextAttributesKey): Array<TextAttributesKey> = arrayOf(*keys)
}
@@ -0,0 +1,25 @@
/*
* Copyright 2025 Sergey S. Chernov real.sergeych@gmail.com
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package net.sergeych.lyng.idea.highlight
import com.intellij.openapi.fileTypes.SingleLazyInstanceSyntaxHighlighterFactory
import com.intellij.openapi.fileTypes.SyntaxHighlighter
class LyngSyntaxHighlighterFactory : SingleLazyInstanceSyntaxHighlighterFactory() {
override fun createHighlighter(): SyntaxHighlighter = LyngSyntaxHighlighter()
}
Loaded 100 of 382 files, more files were not shown because too many files have changed in this diff. Show more