parallelism and flows + bugfixes release

docs/tutorial.md

@@ -16,7 +16,8 @@ __Other documents to read__ maybe after this one:
 - [math in Lyng](math.md)
 - [parallelism] - multithreaded code, coroutines, etc.
 - Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md)
-- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
+- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and
+  loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
 
 # Expressions
 
@@ -171,7 +172,8 @@ allow to improve code look and readability. There are borrowed from Kotlin:
 
 ### let
 
-`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's returned value. It is useful dealing with null or to
+`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's
+returned value. It is useful dealing with null or to
 get a snapshot of some externally varying value, or with `?.` to process nullable value in a safe manner:
 
     // this state is changed from parallel processes
@@ -465,7 +467,6 @@ after function call, it is treated as a last argument to the call, e.g.:
     assert( [11, 21, 31] == mapped)
     >>> void
 
-
 # Lists (aka arrays)
 
 Lyng has built-in mutable array class `List` with simple literals:
@@ -535,7 +536,8 @@ The simplest way to concatenate lists is using `+` and `+=`:
     void
     >>> void
 
-***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will always add all its contents. Use `list.add` to add a single iterable instance:
+***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will
+always add all its contents. Use `list.add` to add a single iterable instance:
 
     var list = [1, 2]
     val other = [3, 4]
@@ -563,7 +565,6 @@ Use `list.add` to avoid confusion:
     assert( list == [1, 2, [3, 4], (10..12)])
     >>> void
 
-
 To add elements to the list:
 
     val x = [1,2]
@@ -596,7 +597,6 @@ Using splat arguments can simplify inserting list in list:
     x
     >>> [1, 0, 100, 0, 2, 3]
 
-
 Note that to add to the end you still need to use `add` or positive index of the after-last element:
 
     val x = [1,2,3]
@@ -742,7 +742,9 @@ Also, you can check the type too:
 
 #### Contains:
 
-You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and `!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be thrown.
+You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and
+`!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be
+thrown.
 
 Typical builtin types that are containers (e.g. support `conain`):
 
@@ -758,7 +760,8 @@ Typical builtin types that are containers (e.g. support `conain`):
 : Iterable is not the container as it can be infinite
 
 (2)
-: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is usually not what you expect of it:
+: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is
+usually not what you expect of it:
 
     assert( "more" in "a".."z") // string range ok
     assert( 'x' !in "a".."z") // char in string range: probably error
@@ -767,7 +770,9 @@ Typical builtin types that are containers (e.g. support `conain`):
     >>> void
 
 (3)
-: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array]. String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it it would be iterable).
+: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array].
+String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it
+it would be iterable).
 
 So we recommend not to mix characters and string ranges; use `ch in str` that works
 as expected:
@@ -845,7 +850,8 @@ We can skip the rest of the loop and restart it, as usual, with `continue` opera
     "found even numbers: " + countEven
     >>> "found even numbers: 5"
 
-`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we intentionally avoid using for loops here):
+`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we
+intentionally avoid using for loops here):
 
     var count = 0
     var total = 0
@@ -891,21 +897,22 @@ test function (remember function return it's last expression result):
 ```mermaid
 flowchart TD
     S((start)) --> Cond{check}
-    Cond--false, no else--->V((void))
+    Cond -- false, no else ---> V((void))
-    Cond--true-->E(["last = loop_body()" ])
+    Cond -- true --> E(["last = loop_body()"])
-    E--break value---->BV((value))
+    E -- break value ----> BV((value))
-    E--> Check2{check}
+    E --> Check2{check}
-    E--break---->V
+    E -- break ----> V
-    Check2 --false-->E
+    Check2 -- false --> E
-    Check2 --true, no else--->L((last))
+    Check2 -- true, no else ---> L((last))
-    Check2 --true, else-->Else(["else_clause()"])
+    Check2 -- true, else --> Else(["else_clause()"])
-    Cond--false, else--->Else
+    Cond -- false, else ---> Else
     Else --> Ele4$nr((else))
 ```
 
 So the returned value, as seen from diagram could be one of:
 
-- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the empty break was executed.
+- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the
+  empty break was executed.
 - value returned from `break value' statement
 - value returned from the `else` clause, of the loop was not broken
 - value returned from the last execution of loop body, if there was no `break` and no `else` clause.
@@ -929,7 +936,8 @@ available in the condition:
     } while( continueLoop )
     >>> "OK"
 
-This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value returning by the loop:
+This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value
+returning by the loop:
 
     fun readLine() { "done: result" }
     val result = do {
@@ -940,7 +948,6 @@ This is sometimes convenient when condition is complex and has to be calculated
 
 Suppose readLine() here reads some stream of lines.
 
-
 ## For loops
 
 For loop are intended to traverse collections, and all other objects that supports
@@ -983,7 +990,8 @@ We can use labels too:
 
 # Exception handling
 
-Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the catch block that caught the exception:
+Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the
+catch block that caught the exception:
 
     var error = "not caught"
     var finallyCaught = false
@@ -1036,7 +1044,6 @@ many more.
 
 - see [exception handling](exceptions_handling.md) for detailed exceptions tutorial and reference.
 
-
 # Self-assignments in expression
 
 There are auto-increments and auto-decrements:
@@ -1159,7 +1166,6 @@ Are the same as in string literals with little difference:
 | code   | Int  | Unicode code for the character |
 |        |      |                                |
 
-
 ## String details
 
 Strings are arrays of Unicode characters. It can be indexed, and indexing will
@@ -1187,7 +1193,10 @@ To format a string use sprintf-style modifiers like:
     assertEquals( "hello :11    ", "%-6s:%-6d"(a, b) )
     >>> void
 
-List of format specifiers closely resembles C sprintf() one. See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
+List of format specifiers closely resembles C sprintf() one.
+See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe
+using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types
+are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
 
 This list will be extended.
 
@@ -1219,7 +1228,7 @@ Concatenation is a `+`: `"hello " + name` works as expected. No confusion.
 Typical set of String functions includes:
 
 | fun/prop           | description / notes                                        |
-|-------------------|------------------------------------------------------------|
+|--------------------|------------------------------------------------------------|
 | lower()            | change case to unicode upper                               |
 | upper()            | change case to unicode lower                               |
 | startsWith(prefix) | true if starts with a prefix                               |
@@ -1242,9 +1251,6 @@ Typical set of String functions includes:
 (1)
 : List is mutable therefore a new copy is created on each call.
 
-
-
-
 ### Literals
 
 String literal could be multiline:
@@ -1259,9 +1265,17 @@ though multiline literals is yet work in progress.
 See [math functions](math.md). Other general purpose functions are:
 
 | name                                       | description                                               |
-|----------------------------------------------|----------------------------------------------------------|
+|--------------------------------------------|-----------------------------------------------------------|
 | assert(condition,message="assertion failed") | runtime code check. There will be an option to skip them  |
-| println(args...)                             | Open for overriding, it prints to stdout.                |
+| assertEquals(a,b)                          |                                                           |
+| assertNotEquals(a,b)                       |                                                           |
+| assertTrows { /* block */ }                |                                                           |
+| check(condition, message=<default>)        | throws IllegalStateException" of condition isn't met      |
+| require(condition, message=<default>)      | throws IllegalArgumentException" of condition isn't met   |
+| println(args...)                           | Open for overriding, it prints to stdout with newline.    |
+| print(args...)                             | Open for overriding, it prints to stdout without newline. |
+| flow {}                                    | create flow sequence, see [parallelism]                   |
+| delay, launch, yield                       | see [parallelism]                                         |
 
 # Built-in constants
 
@@ -1271,13 +1285,23 @@ See [math functions](math.md). Other general purpose functions are:
 | π                                   | See [math](math.md)          |
 
 [List]: List.md
+
 [Iterable]: Iterable.md
+
 [Iterator]: Iterator.md
+
 [Real]: Real.md
+
 [Range]: Range.md
+
 [String]: String.md
+
 [string formatting]: https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary
+
 [Set]: Set.md
+
 [Map]: Map.md
+
 [Buffer]: Buffer.md
+
 [parallelism]: parallelism.md

lynglib/build.gradle.kts

@@ -4,7 +4,7 @@ import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
 import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 
 group = "net.sergeych"
-version = "0.8.2-SNAPSHOT"
+version = "0.8.3-SNAPSHOT"
 
 buildscript {
     repositories {

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Scope.kt

@@ -205,6 +205,20 @@ open class Scope(
         return "S[this=$thisObj $contents]"
     }
 
+    fun trace(text: String="") {
+        println("trace Scope: $text ------------------")
+        var p = this.parent
+        var level = 0
+        while (p != null) {
+            println("     parent#${++level}: $p")
+            println("     ( ${p.args.list} )")
+            p = p.parent
+        }
+        println("--------------------")
+        ObjVoid
+
+    }
+
     companion object {
 
         fun new(): Scope =

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt

@@ -26,8 +26,15 @@ class Script(
 
     companion object {
 
-        private val rootScope: Scope = Scope(null).apply {
+        internal val rootScope: Scope = Scope(null).apply {
             ObjException.addExceptionsToContext(this)
+            addFn("print") {
+                for ((i, a) in args.withIndex()) {
+                    if (i > 0) print(' ' + a.asStr.value)
+                    else print(a.asStr.value)
+                }
+                ObjVoid
+            }
             addFn("println") {
                 for ((i, a) in args.withIndex()) {
                     if (i > 0) print(' ' + a.asStr.value)
@@ -153,16 +160,26 @@ class Script(
                 }
                 result ?: raiseError(ObjAssertionFailedException(this,"Expected exception but nothing was thrown"))
             }
-            addFn("traceScope") {
+            addFn("require") {
-                println("trace Scope: $this")
+                val condition = requiredArg<ObjBool>(0)
-                var p = this.parent
+                if( !condition.value ) {
-                var level = 0
+                    val message = args.list.getOrNull(1)?.toString() ?: "requirement not met"
-                while (p != null) {
+                    raiseIllegalArgument(message)
-                    println("     parent#${++level}: $p")
-                    p = p.parent
                 }
                 ObjVoid
             }
+            addFn("check") {
+                val condition = requiredArg<ObjBool>(0)
+                if( !condition.value ) {
+                    val message = args.list.getOrNull(1)?.toString() ?: "check failed"
+                    raiseIllegalState(message)
+                }
+                ObjVoid
+            }
+            addFn("traceScope") {
+                this.trace(args.getOrNull(0)?.toString() ?: "")
+                ObjVoid
+            }
 
             addVoidFn("delay") {
                 delay((this.args.firstAndOnly().toDouble()/1000.0).roundToLong())
@@ -204,7 +221,9 @@ class Script(
             }
 
             addFn("flow") {
-                ObjFlow(requireOnlyArg<Statement>())
+                // important is: current context contains closure often used in call;
+                // we'll need it for the producer
+                ObjFlow(requireOnlyArg<Statement>(), this)
             }
 
             val pi = ObjReal(PI)

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjFlow.kt

@@ -7,9 +7,7 @@ import kotlinx.coroutines.channels.ReceiveChannel
 import kotlinx.coroutines.channels.SendChannel
 import kotlinx.coroutines.sync.Mutex
 import kotlinx.coroutines.sync.withLock
-import net.sergeych.lyng.Scope
+import net.sergeych.lyng.*
-import net.sergeych.lyng.ScriptFlowIsNoMoreCollected
-import net.sergeych.lyng.Statement
 import net.sergeych.mp_tools.globalLaunch
 import kotlin.coroutines.cancellation.CancellationException
 
@@ -60,7 +58,7 @@ private fun createLyngFlowInput(scope: Scope, producer: Statement): ReceiveChann
     return channel
 }
 
-class ObjFlow(val producer: Statement) : Obj() {
+class ObjFlow(val producer: Statement, val scope: Scope) : Obj() {
 
     override val objClass = type
 
@@ -71,7 +69,8 @@ class ObjFlow(val producer: Statement) : Obj() {
             }
         }.apply {
             addFn("iterator") {
-                ObjFlowIterator(thisAs<ObjFlow>().producer)
+                val objFlow = thisAs<ObjFlow>()
+                ObjFlowIterator( statement { objFlow.producer.execute(ClosureScope(this,objFlow.scope)) } )
             }
         }
     }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjIterable.kt

@@ -111,6 +111,5 @@ val ObjIterable by lazy {
                     .not()
             )
         }
-
     }
 }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/stdlib_included/root_lyng.kt

@@ -0,0 +1,5 @@
+package net.sergeych.lyng.stdlib_included
+
+internal val rootLyng = """
+    
+""".trimIndent()

lynglib/src/commonMain/lyng/stdlib/Iterable.lyng

@@ -0,0 +1,16 @@
+
+fun Iterable.filter( predicate ) {
+    flow {
+        for( item in this )
+            if( predicate(item) )
+                emit(item)
+    }
+}
+
+fun Iterable.drop(n) {
+    require( n >= 0, "drop amount must be non-negative")
+    var count = 0
+    filter {
+        count++ < N
+    }
+}

lynglib/src/commonTest/kotlin/CoroutinesTest.kt

@@ -106,4 +106,67 @@ class TestCoroutines {
                 assertEquals( result, f.toList())
         """.trimIndent())
     }
+
+    @Test
+    fun testFlowClosures() = runTest {
+        eval("""
+            fun filter( a, b ) {
+                println("filter: %s, %s"(a,b))
+                flow {
+                    emit(a)
+                    emit(b)
+                }
+            }
+            
+            assertEquals( [5, 1], filter(5,1).toList() )
+            assertEquals( [2, 3], filter(2,3).toList() )
+            
+        """.trimIndent())
+    }
+
+
+    @Test
+    fun testFilterFlow() = runTest {
+        eval("""
+            fun filter( list, predicate ) {
+                val p = predicate
+                println("predicate "+predicate+" / "+p)
+                flow {
+                    // here p is captured only once and does not change!
+                    for( item in list ) {
+                        print("filter "+p+" "+item+": ")
+                        if( p(item) ) {
+                            println("OK")
+                            emit(item)
+                        }
+                        else println("NO")
+                    }
+                }
+            }
+            
+//            fun drop(i, n) {
+//                require( n >= 0, "drop amount must be non-negative")
+//                var count = 0
+//                println("drop %d"(n))
+//                filter(i) {
+//                    count++ >= n
+//                }
+//            }
+            
+            val src = (1..1).toList()
+            assertEquals( 1, filter(src) { true }.toList().size )
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            assertEquals( 0, filter(src) { false }.toList().size )
+//            assertEquals( 3, filter(src) { true }.size() ) 
+            
+//            assertEquals( [7,8], drop((1..8).toList(),6).toList())
+//            assertEquals( [1,3,5,7], filter((1..8).toList()) { 
+//                println("call2")
+//                it % 2 == 1 
+//            }.toList())
+        """.trimIndent())
+    }
 }