more docs on Iterable

docs/Iterable.md

@@ -1,5 +1,11 @@
 # Iterable interface
 
+The interface for anything that can be iterated, e.g. finite or infinite ordered set of data that can be accessed sequentially. Almost any data container in `Lyng` implements it: `List`, `Set`, `Buffer`, `RingBuffer`, `BitBuffer`, `Range` and many others are `Iterable`, also `Collection` and `Array` interfaces inherit it.
+
+`Map` and `String` have `Iterable` members to access its contents too.
+
+## Definition:
+
 Iterable is a class that provides function that creates _the iterator_:
 
     class Iterable {
@@ -17,9 +23,16 @@ Iterator itself is a simple interface that should provide only to method:
 
 Just remember at this stage typed declarations are not yet supported.
 
-Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions available:
+Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions available, for example
 
-## Instance methods
+    val r = 1..10  // Range is Iterable!  
+    assertEquals( [9,10] r.takeLast(2) )
+    assertEquals( [1,2,3] r.take(3) )
+    assertEquals( [9,10] r.drop(8) )
+    assertEquals( [1,2] r.dropLast(8) )
+    >>> void
+
+## Instance methods:
 
 | fun/method      | description                                                                     |
 |-----------------|---------------------------------------------------------------------------------|
@@ -33,7 +46,15 @@ Having `Iterable` in base classes allows to use it in for loop. Also, each `Iter
 | map(f)          | create a list of values returned by `f` called for each element of the iterable |
 | indexOf(i)      | return index if the first encounter of i or a negative value if not found       |
 | associateBy(kf) | create a map where keys are returned by kf that will be called for each element |
+| first           | first element (1)                                                               |
+| last            | last element (1)                                                                |
+| take(n)         | return [Iterable] of up to n first elements                                     |
+| taleLast(n)     | return [Iterable] of up to n last elements                                      |
+| drop(n)         | return new [Iterable] without first n elements                                  |
+| dropLast(n)     | return new [Iterable] without last n elements                                   |
 
+(1)
+: throws `NoSuchElementException` if there is no such element 
 
     fun Iterable.toList(): List
     fun Iterable.toSet(): Set
@@ -45,7 +66,7 @@ Having `Iterable` in base classes allows to use it in for loop. Also, each `Iter
     fun Iterable.associateBy( keyMaker: (Any?)->Any): Map
     
 
-## Abstract methods
+## Abstract methods:
 
     fun iterator(): Iterator
 
@@ -57,7 +78,9 @@ Creates a list by iterating to the end. So, the Iterator should be finite to be
 
 ## Implemented in classes:
 
-- [List], [Range]
+- [List], [Range], [Buffer](Buffer.md), [BitBuffer], [Buffer], [Set], [RingBuffer]
 
 [List]: List.md
 [Range]: Range.md
+[Set]: Set.md
+[RingBuffer]: RingBuffer.md

docs/RingBuffer.md

@@ -0,0 +1,52 @@
+# RingBuffer
+
+This is a fixed size buffer that allow to store N last elements with _O(1)_ effectiveness (no data shifting).
+
+Here is the sample:
+
+    val r = RingBuffer(3)
+    assert( r is RingBuffer )
+    assertEquals(0, r.size)
+    assertEquals(3, r.capacity)
+    
+    r += 10
+    assertEquals(1, r.size)
+    assertEquals(10, r.first)
+    
+    r += 20
+    assertEquals(2, r.size)
+    assertEquals( [10, 20], r.toList() )
+    
+    r += 30
+    assertEquals(3, r.size)
+    assertEquals( [10, 20, 30], r.toList() )
+    
+    // now first value is lost:
+    r += 40
+    assertEquals(3, r.size)
+    assertEquals( [20, 30, 40], r.toList() )
+    assertEquals(3, r.capacity)
+
+    >>> void
+
+Ring buffer implements [Iterable], so any of its methods are available for `RingBuffer`, e.g. `first`, `last`, `toList`,
+`take`, `drop`, `takelast`, `dropLast`, etc.
+
+## Constructor
+
+    RinbBuffer(capacity: Int)
+
+## Instance methods
+
+| method      | description            | remarks |
+|-------------|------------------------|---------|
+| capacity    | max size of the buffer |         |
+| size        | current size           | (1)     |
+| operator += | add new item           | (1)     |
+| add(item)   | add new item           | (1)     |
+| iterator()  | return iterator        | (1)     |
+
+(1)
+: Ringbuffer is not threadsafe, protect it with a mutex to avoid RC where necessary.
+
+[Iterable]: Iterable.md

docs/parallelism.md

@@ -128,3 +128,80 @@ Usage example:
             yield()
         } while(true)
     }
+
+# Data exchange for coroutines
+
+## Flow
+
+Flow is an async cold sequence; it is named after kotlin's Flow as it resembles it closely. The cold means the flow is only evaluated when iterated (collected, in Kotlin terms), before it is inactive. Sequence means that it is potentially unlimited, as in our example of glorious Fibonacci number generator:
+
+    // Fibonacch numbers flow! 
+    val f = flow {
+        println("Starting generator")
+        var n1 = 0
+        var n2 = 1
+        emit(n1)
+        emit(n2)
+        while(true) {
+            val n = n1 + n2
+            emit(n)
+            n1 = n2
+            n2 = n
+        }
+    }
+    val correctFibs = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765]
+    println("Generation starts")
+    assertEquals( correctFibs, f.take(correctFibs.size))
+    >>> Generation starts
+    >>> Starting generator
+    >>> void
+
+Great: the generator is not executed until collected bu the `f.take()` call, which picks specified number of elements from the flow, can cancel it.
+
+Important difference from the channels or like, every time you collect the flow, you collect it anew:
+
+    var isStarted = false
+    val f = flow {
+            emit("start")
+            isStarted = true
+            (1..4).forEach { emit(it) }
+    }
+    // flow is not yet started, e.g. not got execited, 
+    // that is called 'cold':
+    assertEquals( false, isStarted )
+
+    // let's collect flow:
+    val result = []
+    for( x in f ) result += x
+    println(result)
+
+    assertEquals( true, isStarted)
+
+    // let's collect it once again, it should be the same:
+    println(f.toList())
+
+    // and again:
+    assertEquals( result, f.toList() )
+
+    >>> ["start", 1, 2, 3, 4]
+    >>> ["start", 1, 2, 3, 4]
+    >>> void
+
+Notice that flow's lambda is not called until actual collection is started. Cold flows are
+better in terms of resource consumption.
+
+Flows allow easy transforming of any [Iterable]. See how the standard Lyng library functions use it:
+
+    fun Iterable.filter(predicate) {
+        val list = this
+        flow {
+            for( item in list ) {
+                if( predicate(item) ) {
+                    emit(item)
+                }
+            }
+        }
+    }
+
+
+[Iterable]: Iterable.md

docs/tutorial.md

@@ -14,9 +14,11 @@ __Other documents to read__ maybe after this one:
 - [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md)
 - [OOP notes](OOP.md), [exception handling](exceptions_handling.md)
 - [math in Lyng](math.md)
+- [time](time.md) and [parallelism](parallelism.md)
 - [parallelism] - multithreaded code, coroutines, etc.
-- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md)
+- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md), [RingBuffer], [Buffer].
-- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
+- Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and
+  loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
 
 # Expressions
 
@@ -171,7 +173,8 @@ allow to improve code look and readability. There are borrowed from Kotlin:
 
 ### let
 
-`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's returned value. It is useful dealing with null or to
+`value.let {}` passes to the block value as the single parameter (by default it is assigned to `it`) and return block's
+returned value. It is useful dealing with null or to
 get a snapshot of some externally varying value, or with `?.` to process nullable value in a safe manner:
 
     // this state is changed from parallel processes
@@ -465,7 +468,6 @@ after function call, it is treated as a last argument to the call, e.g.:
     assert( [11, 21, 31] == mapped)
     >>> void
 
-
 # Lists (aka arrays)
 
 Lyng has built-in mutable array class `List` with simple literals:
@@ -535,7 +537,8 @@ The simplest way to concatenate lists is using `+` and `+=`:
     void
     >>> void
 
-***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will always add all its contents. Use `list.add` to add a single iterable instance:
+***Important note***: the pitfall of using `+=` is that you can't append in [Iterable] instance as an object: it will
+always add all its contents. Use `list.add` to add a single iterable instance:
 
     var list = [1, 2]
     val other = [3, 4]
@@ -563,7 +566,6 @@ Use `list.add` to avoid confusion:
     assert( list == [1, 2, [3, 4], (10..12)])
     >>> void
 
-
 To add elements to the list:
 
     val x = [1,2]
@@ -596,7 +598,6 @@ Using splat arguments can simplify inserting list in list:
     x
     >>> [1, 0, 100, 0, 2, 3]
 
-
 Note that to add to the end you still need to use `add` or positive index of the after-last element:
 
     val x = [1,2,3]
@@ -742,7 +743,9 @@ Also, you can check the type too:
 
 #### Contains:
 
-You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and `!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be thrown.
+You can thest that _when expression_ is _contained_, or not contained, in some object using `in container` and
+`!in container`. The container is any object that provides `contains` method, otherwise the runtime exception will be
+thrown.
 
 Typical builtin types that are containers (e.g. support `conain`):
 
@@ -758,7 +761,8 @@ Typical builtin types that are containers (e.g. support `conain`):
 : Iterable is not the container as it can be infinite
 
 (2)
-: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is usually not what you expect of it:
+: Depending on the inclusivity and open/closed range parameters. BE careful here: String range is allowed, but it is
+usually not what you expect of it:
 
     assert( "more" in "a".."z") // string range ok
     assert( 'x' !in "a".."z") // char in string range: probably error
@@ -767,7 +771,9 @@ Typical builtin types that are containers (e.g. support `conain`):
     >>> void
 
 (3)
-: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array]. String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it it would be iterable).
+: `String` also can provide array of characters directly with `str.characters()`, which is [Iterable] and [Array].
+String itself is not iterable as otherwise it will interfere when adding strigns to lists (it will add _characters_ it
+it would be iterable).
 
 So we recommend not to mix characters and string ranges; use `ch in str` that works
 as expected:
@@ -845,7 +851,8 @@ We can skip the rest of the loop and restart it, as usual, with `continue` opera
     "found even numbers: " + countEven
     >>> "found even numbers: 5"
 
-`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we intentionally avoid using for loops here):
+`continue` can't "return" anything: it just restarts the loop. It can use labeled loops to restart outer ones (we
+intentionally avoid using for loops here):
 
     var count = 0
     var total = 0
@@ -891,21 +898,22 @@ test function (remember function return it's last expression result):
 ```mermaid
 flowchart TD
     S((start)) --> Cond{check}
-    Cond--false, no else--->V((void))
+    Cond -- false, no else ---> V((void))
-    Cond--true-->E(["last = loop_body()" ])
+    Cond -- true --> E(["last = loop_body()"])
-    E--break value---->BV((value))
+    E -- break value ----> BV((value))
-    E--> Check2{check}
+    E --> Check2{check}
-    E--break---->V
+    E -- break ----> V
-    Check2 --false-->E
+    Check2 -- false --> E
-    Check2 --true, no else--->L((last))
+    Check2 -- true, no else ---> L((last))
-    Check2 --true, else-->Else(["else_clause()"])
+    Check2 -- true, else --> Else(["else_clause()"])
-    Cond--false, else--->Else
+    Cond -- false, else ---> Else
     Else --> Ele4$nr((else))
 ```
 
 So the returned value, as seen from diagram could be one of:
 
-- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the empty break was executed.
+- `void`, if the loop was not executed, e.g. `condition` was initially false, and there was no `else` clause, or if the
+  empty break was executed.
 - value returned from `break value' statement
 - value returned from the `else` clause, of the loop was not broken
 - value returned from the last execution of loop body, if there was no `break` and no `else` clause.
@@ -929,7 +937,8 @@ available in the condition:
     } while( continueLoop )
     >>> "OK"
 
-This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value returning by the loop:
+This is sometimes convenient when condition is complex and has to be calculated inside the loop body. Notice the value
+returning by the loop:
 
     fun readLine() { "done: result" }
     val result = do {
@@ -940,7 +949,6 @@ This is sometimes convenient when condition is complex and has to be calculated
 
 Suppose readLine() here reads some stream of lines.
 
-
 ## For loops
 
 For loop are intended to traverse collections, and all other objects that supports
@@ -983,7 +991,8 @@ We can use labels too:
 
 # Exception handling
 
-Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the catch block that caught the exception:
+Very much like in Kotlin. Try block returns its body block result, if no exception was cauht, or the result from the
+catch block that caught the exception:
 
     var error = "not caught"
     var finallyCaught = false
@@ -1036,7 +1045,6 @@ many more.
 
 - see [exception handling](exceptions_handling.md) for detailed exceptions tutorial and reference.
 
-
 # Self-assignments in expression
 
 There are auto-increments and auto-decrements:
@@ -1159,7 +1167,6 @@ Are the same as in string literals with little difference:
 | code   | Int  | Unicode code for the character |
 |        |      |                                |
 
-
 ## String details
 
 Strings are arrays of Unicode characters. It can be indexed, and indexing will
@@ -1187,7 +1194,10 @@ To format a string use sprintf-style modifiers like:
     assertEquals( "hello :11    ", "%-6s:%-6d"(a, b) )
     >>> void
 
-List of format specifiers closely resembles C sprintf() one. See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
+List of format specifiers closely resembles C sprintf() one.
+See [format specifiers](https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary), this is doe
+using [mp_stools kotlin multiplatform library](https://github.com/sergeych/mp_stools). Currently supported Lyng types
+are `String`, `Int`, `Real`, `Bool`, the rest are displayed using their `toString()` representation.
 
 This list will be extended.
 
@@ -1219,7 +1229,7 @@ Concatenation is a `+`: `"hello " + name` works as expected. No confusion.
 Typical set of String functions includes:
 
 | fun/prop           | description / notes                                        |
-|-------------------|------------------------------------------------------------|
+|--------------------|------------------------------------------------------------|
 | lower()            | change case to unicode upper                               |
 | upper()            | change case to unicode lower                               |
 | startsWith(prefix) | true if starts with a prefix                               |
@@ -1242,9 +1252,6 @@ Typical set of String functions includes:
 (1)
 : List is mutable therefore a new copy is created on each call.
 
-
-
-
 ### Literals
 
 String literal could be multiline:
@@ -1259,9 +1266,17 @@ though multiline literals is yet work in progress.
 See [math functions](math.md). Other general purpose functions are:
 
 | name                                       | description                                               |
-|----------------------------------------------|----------------------------------------------------------|
+|--------------------------------------------|-----------------------------------------------------------|
 | assert(condition,message="assertion failed") | runtime code check. There will be an option to skip them  |
-| println(args...)                             | Open for overriding, it prints to stdout.                |
+| assertEquals(a,b)                          |                                                           |
+| assertNotEquals(a,b)                       |                                                           |
+| assertTrows { /* block */ }                |                                                           |
+| check(condition, message=<default>)        | throws IllegalStateException" of condition isn't met      |
+| require(condition, message=<default>)      | throws IllegalArgumentException" of condition isn't met   |
+| println(args...)                           | Open for overriding, it prints to stdout with newline.    |
+| print(args...)                             | Open for overriding, it prints to stdout without newline. |
+| flow {}                                    | create flow sequence, see [parallelism]                   |
+| delay, launch, yield                       | see [parallelism]                                         |
 
 # Built-in constants
 
@@ -1271,13 +1286,25 @@ See [math functions](math.md). Other general purpose functions are:
 | π                                   | See [math](math.md)          |
 
 [List]: List.md
+
 [Iterable]: Iterable.md
+
 [Iterator]: Iterator.md
+
 [Real]: Real.md
+
 [Range]: Range.md
+
 [String]: String.md
+
 [string formatting]: https://github.com/sergeych/mp_stools?tab=readme-ov-file#sprintf-syntax-summary
+
 [Set]: Set.md
+
 [Map]: Map.md
+
 [Buffer]: Buffer.md
+
 [parallelism]: parallelism.md
+
+[RingBuffer]: RingBuffer.md

lyng/src/commonMain/kotlin/Common.kt

@@ -10,10 +10,11 @@ import com.github.ajalt.clikt.parameters.options.flag
 import com.github.ajalt.clikt.parameters.options.option
 import kotlinx.coroutines.runBlocking
 import net.sergeych.lyng.LyngVersion
-import net.sergeych.lyng.Scope
+import net.sergeych.lyng.Script
 import net.sergeych.lyng.ScriptError
 import net.sergeych.lyng.Source
 import net.sergeych.lyng.obj.*
+import net.sergeych.mp_tools.globalDefer
 import okio.FileSystem
 import okio.Path.Companion.toPath
 import okio.SYSTEM
@@ -38,14 +39,13 @@ data class CommandResult(
     val error: String
 )
 
-val baseScope = Scope().apply {
+val baseScopeDefer = globalDefer {
+    Script.newScope().apply {
         addFn("exit") {
             exit(requireOnlyArg<ObjInt>().toInt())
             ObjVoid
         }
-//    ObjString.type.addFn("shell") {
+    }
-//
-//    }
 }
 
 fun runMain(args: Array<String>) {
@@ -88,6 +88,8 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
         """.trimIndent()
 
     override fun run() {
+        runBlocking {
+            val baseScope = baseScopeDefer.await()
             when {
                 version -> {
                     println("Lyng language version ${LyngVersion}")
@@ -127,11 +129,12 @@ private class Lyng(val launcher: (suspend () -> Unit) -> Unit) : CliktCommand()
                 }
             }
         }
+    }
 }
 
 fun executeFileWithArgs(fileName: String, args: List<String>) {
     runBlocking {
-        baseScope.addConst("ARGV", ObjList(args.map { ObjString(it) }.toMutableList()))
+        baseScopeDefer.await().addConst("ARGV", ObjList(args.map { ObjString(it) }.toMutableList()))
         executeFile(fileName)
     }
 }
@@ -148,7 +151,7 @@ suspend fun executeFile(fileName: String) {
         text = text.substring(pos + 1)
     }
     processErrors {
-        baseScope.eval(Source(fileName, text))
+        baseScopeDefer.await().eval(Source(fileName, text))
     }
 }
 

lynglib/build.gradle.kts

@@ -4,7 +4,7 @@ import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
 import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 
 group = "net.sergeych"
-version = "0.8.2-SNAPSHOT"
+version = "0.8.4-SNAPSHOT"
 
 buildscript {
     repositories {
@@ -65,6 +65,7 @@ kotlin {
             languageSettings.optIn("kotlin.contracts.ExperimentalContracts")
             languageSettings.optIn("kotlin.ExperimentalUnsignedTypes")
             languageSettings.optIn("kotlin.coroutines.DelicateCoroutinesApi")
+            languageSettings.optIn("kotlinx.coroutines.flow.DelicateCoroutinesApi")
         }
 
         val commonMain by getting {

lynglib/src/commonMain/kotlin/net/sergeych/lyng/AppliedScope.kt

@@ -1,19 +0,0 @@
-package net.sergeych.lyng
-
-import net.sergeych.lyng.obj.ObjRecord
-
-/**
- * Special version of the [Scope] used to `apply` new this object to
- * _parent context property.
- *
- * @param _parent context to apply to
- * @param args arguments for the new context
- * @param appliedScope the new context to apply, it will have lower priority except for `this` which
- *      will be reset by appliedContext's `this`.
- */
-class AppliedScope(_parent: Scope, args: Arguments, val appliedScope: Scope)
-    : Scope(_parent, args, appliedScope.pos, appliedScope.thisObj) {
-    override fun get(name: String): ObjRecord? =
-        if (name == "this") thisObj.asReadonly
-        else super.get(name) ?: appliedScope[name]
-}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/ArgsDeclaration.kt

@@ -34,7 +34,8 @@ data class ArgsDeclaration(val params: List<Item>, val endTokenType: Token.Type)
         defaultRecordType: ObjRecord.Type = ObjRecord.Type.ConstructorField
     ) {
         fun assign(a: Item, value: Obj) {
-            scope.addItem(a.name, (a.accessType ?: defaultAccessType).isMutable, value,
+            scope.addItem(a.name, (a.accessType ?: defaultAccessType).isMutable,
+                value.byValueCopy(),
                 a.visibility ?: defaultVisibility,
                 recordType = defaultRecordType)
         }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Arguments.kt

@@ -36,7 +36,7 @@ data class Arguments(val list: List<Obj>, val tailBlockMode: Boolean = false) :
 
     fun firstAndOnly(pos: Pos = Pos.UNKNOWN): Obj {
         if (list.size != 1) throw ScriptError(pos, "expected one argument, got ${list.size}")
-        return list.first()
+        return list.first().byValueCopy()
     }
 
     /**

lynglib/src/commonMain/kotlin/net/sergeych/lyng/ClosureScope.kt

@@ -0,0 +1,16 @@
+package net.sergeych.lyng
+
+import net.sergeych.lyng.obj.ObjRecord
+
+/**
+ * Scope that adds a "closure" to caller; most often it is used to apply class instance to caller scope.
+ * Inherits [Scope.args] and [Scope.thisObj] from [callScope] and adds lookup for symbols
+ * from [closureScope] with proper precedence
+ */
+class ClosureScope(val callScope: Scope,val closureScope: Scope) : Scope(callScope, callScope.args, thisObj = callScope.thisObj) {
+
+    override fun get(name: String): ObjRecord? {
+        // closure should be treated below callScope
+        return super.get(name) ?: closureScope.get(name)
+    }
+}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Compiler.kt

@@ -470,7 +470,7 @@ class Compiler(
 
         val callStatement = statement {
             // and the source closure of the lambda which might have other thisObj.
-            val context = AppliedScope(closure!!, args, this)
+            val context = ClosureScope(this, closure!!) //AppliedScope(closure!!, args, this)
             if (argsDeclaration == null) {
                 // no args: automatic var 'it'
                 val l = args.list
@@ -1540,7 +1540,8 @@ class Compiler(
         }
     }
 
-    private suspend fun parseFunctionDeclaration(
+    private suspend fun
+        parseFunctionDeclaration(
         visibility: Visibility = Visibility.Public,
         @Suppress("UNUSED_PARAMETER") isOpen: Boolean = false,
         isExtern: Boolean = false,
@@ -1586,9 +1587,11 @@ class Compiler(
 
         val fnBody = statement(t.pos) { callerContext ->
             callerContext.pos = start
+
             // restore closure where the function was defined, and making a copy of it
             // for local space (otherwise it will write local stuff to closure!)
-            val context = closure?.copy() ?: callerContext.raiseError("bug: closure not set")
+            val context = closure?.let { ClosureScope(callerContext, it) }
+                ?: callerContext.raiseError("bug: closure not set")
 
             // load params from caller context
             argsDeclaration.assignToContext(context, callerContext.args, defaultAccessType = AccessType.Val)
@@ -1597,7 +1600,7 @@ class Compiler(
             }
             fnStatements.execute(context)
         }
-        val fnCreatestatement = statement(start) { context ->
+        val fnCreateStatement = statement(start) { context ->
             // we added fn in the context. now we must save closure
             // for the function
             closure = context
@@ -1606,7 +1609,12 @@ class Compiler(
                 val type = context[typeName]?.value ?: context.raiseSymbolNotFound("class $typeName not found")
                 if (type !is ObjClass) context.raiseClassCastError("$typeName is not the class instance")
                 type.addFn(name, isOpen = true) {
-                    fnBody.execute(this)
+                    // ObjInstance has a fixed instance scope, so we need to build a closure
+                    (thisObj as? ObjInstance)?.let { i ->
+                        fnBody.execute(ClosureScope(this, i.instanceScope))
+                    }
+                        // other classes can create one-time scope for this rare case:
+                        ?: fnBody.execute(thisObj.autoInstanceScope(this))
                 }
             }
             // regular function/method
@@ -1616,10 +1624,10 @@ class Compiler(
             fnBody
         }
         return if (isStatic) {
-            currentInitScope += fnCreatestatement
+            currentInitScope += fnCreateStatement
             NopStatement
         } else
-            fnCreatestatement
+            fnCreateStatement
     }
 
     private suspend fun parseBlock(skipLeadingBrace: Boolean = false): Statement {

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Scope.kt

@@ -16,7 +16,7 @@ import net.sergeych.lyng.pacman.ImportProvider
  *
  *  There are special types of scopes:
  *
- *  - [AppliedScope] - scope used to apply a closure to some thisObj scope
+ *  - [ClosureScope] - scope used to apply a closure to some thisObj scope
  */
 open class Scope(
     val parent: Scope?,
@@ -73,7 +73,7 @@ open class Scope(
 
     inline fun <reified T : Obj> requiredArg(index: Int): T {
         if (args.list.size <= index) raiseError("Expected at least ${index + 1} argument, got ${args.list.size}")
-        return (args.list[index] as? T)
+        return (args.list[index].byValueCopy() as? T)
             ?: raiseClassCastError("Expected type ${T::class.simpleName}, got ${args.list[index]::class.simpleName}")
     }
 
@@ -94,8 +94,15 @@ open class Scope(
             raiseError("This function does not accept any arguments")
     }
 
-    inline fun <reified T : Obj> thisAs(): T = (thisObj as? T)
+    inline fun <reified T : Obj> thisAs(): T {
-        ?: raiseClassCastError("Cannot cast ${thisObj.objClass.className} to ${T::class.simpleName}")
+        var s: Scope? = this
+        do {
+            val t = s!!.thisObj
+            if (t is T) return t
+            s = s.parent
+        } while(s != null)
+        raiseClassCastError("Cannot cast ${thisObj.objClass.className} to ${T::class.simpleName}")
+    }
 
     internal val objects = mutableMapOf<String, ObjRecord>()
 
@@ -156,13 +163,14 @@ open class Scope(
     fun addConst(name: String, value: Obj) = addItem(name, false, value)
 
     suspend fun eval(code: String): Obj =
-        Compiler.compile(code.toSource(), currentImportProvider).execute(this)
+        eval(code.toSource())
 
-    suspend fun eval(source: Source): Obj =
+    suspend fun eval(source: Source): Obj {
-        Compiler.compile(
+        return Compiler.compile(
             source,
             currentImportProvider
         ).execute(this)
+    }
 
     fun containsLocal(name: String): Boolean = name in objects
 
@@ -193,6 +201,25 @@ open class Scope(
     val importManager by lazy { (currentImportProvider as? ImportManager)
         ?: throw IllegalStateException("this scope has no manager in the chain (provided $currentImportProvider") }
 
+    override fun toString(): String {
+        val contents = objects.entries.joinToString { "${if( it.value.isMutable ) "var" else "val" } ${it.key}=${it.value.value}" }
+        return "S[this=$thisObj $contents]"
+    }
+
+    fun trace(text: String="") {
+        println("trace Scope: $text ------------------")
+        var p = this.parent
+        var level = 0
+        while (p != null) {
+            println("     parent#${++level}: $p")
+            println("     ( ${p.args.list} )")
+            p = p.parent
+        }
+        println("--------------------")
+        ObjVoid
+
+    }
+
     companion object {
 
         fun new(): Scope =

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt

@@ -4,6 +4,7 @@ import kotlinx.coroutines.delay
 import kotlinx.coroutines.yield
 import net.sergeych.lyng.obj.*
 import net.sergeych.lyng.pacman.ImportManager
+import net.sergeych.lyng.stdlib_included.rootLyng
 import net.sergeych.lynon.ObjLynonClass
 import net.sergeych.mp_tools.globalDefer
 import kotlin.math.*
@@ -22,12 +23,23 @@ class Script(
         return lastResult
     }
 
-    suspend fun execute() = execute(defaultImportManager.newModule())
+    suspend fun execute() = execute(
+        defaultImportManager.newStdScope()
+    )
 
     companion object {
 
-        private val rootScope: Scope = Scope(null).apply {
+        suspend fun newScope(pos: Pos = Pos.builtIn) = defaultImportManager.newStdScope(pos)
+
+        internal val rootScope: Scope = Scope(null).apply {
             ObjException.addExceptionsToContext(this)
+            addFn("print") {
+                for ((i, a) in args.withIndex()) {
+                    if (i > 0) print(' ' + a.asStr.value)
+                    else print(a.asStr.value)
+                }
+                ObjVoid
+            }
             addFn("println") {
                 for ((i, a) in args.withIndex()) {
                     if (i > 0) print(' ' + a.asStr.value)
@@ -153,6 +165,26 @@ class Script(
                 }
                 result ?: raiseError(ObjAssertionFailedException(this,"Expected exception but nothing was thrown"))
             }
+            addFn("require") {
+                val condition = requiredArg<ObjBool>(0)
+                if( !condition.value ) {
+                    val message = args.list.getOrNull(1)?.toString() ?: "requirement not met"
+                    raiseIllegalArgument(message)
+                }
+                ObjVoid
+            }
+            addFn("check") {
+                val condition = requiredArg<ObjBool>(0)
+                if( !condition.value ) {
+                    val message = args.list.getOrNull(1)?.toString() ?: "check failed"
+                    raiseIllegalState(message)
+                }
+                ObjVoid
+            }
+            addFn("traceScope") {
+                this.trace(args.getOrNull(0)?.toString() ?: "")
+                ObjVoid
+            }
 
             addVoidFn("delay") {
                 delay((this.args.firstAndOnly().toDouble()/1000.0).roundToLong())
@@ -175,6 +207,7 @@ class Script(
             addConst("Iterable", ObjIterable)
             addConst("Collection", ObjCollection)
             addConst("Array", ObjArray)
+            addConst("RingBuffer", ObjRingBuffer.type)
             addConst("Class", ObjClassType)
 
             addConst("Deferred", ObjDeferred.type)
@@ -182,7 +215,7 @@ class Script(
             addConst("Mutex", ObjMutex.type)
 
             addFn("launch") {
-                val callable = args.firstAndOnly() as Statement
+                val callable = requireOnlyArg<Statement>()
                 ObjDeferred(globalDefer {
                     callable.execute(this@addFn)
                 })
@@ -193,6 +226,11 @@ class Script(
                 ObjVoid
             }
 
+            addFn("flow") {
+                // important is: current context contains closure often used in call;
+                // we'll need it for the producer
+                ObjFlow(requireOnlyArg<Statement>(), this)
+            }
 
             val pi = ObjReal(PI)
             addConst("π", pi)
@@ -203,6 +241,9 @@ class Script(
 
         val defaultImportManager: ImportManager by lazy {
             ImportManager(rootScope, SecurityManager.allowAll).apply {
+                addTextPackages(
+                    rootLyng
+                )
                 addPackage("lyng.buffer") {
                     it.addConst("Buffer", ObjBuffer.type)
                     it.addConst("MutableBuffer", ObjMutableBuffer.type)

lynglib/src/commonMain/kotlin/net/sergeych/lyng/ScriptError.kt

@@ -14,6 +14,8 @@ open class ScriptError(val pos: Pos, val errorMessage: String, cause: Throwable?
     cause
 )
 
+class ScriptFlowIsNoMoreCollected: Exception()
+
 class ExecutionError(val errorObject: ObjException) : ScriptError(errorObject.scope.pos, errorObject.message)
 
 class ImportException(pos: Pos, message: String) : ScriptError(pos, message)

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/Obj.kt

@@ -55,7 +55,8 @@ open class Obj {
         scope: Scope,
         name: String,
         args: Arguments = Arguments.EMPTY
-    ): T = invokeInstanceMethod(scope, name, args) as T
+    ): T =
+        invokeInstanceMethod(scope, name, args) as T
 
     /**
      * Invoke a method of the object if exists
@@ -68,7 +69,10 @@ open class Obj {
         args: Arguments = Arguments.EMPTY,
         onNotFoundResult: Obj?=null
     ): Obj =
-        objClass.getInstanceMemberOrNull(name)?.value?.invoke(scope, this, args)
+        objClass.getInstanceMemberOrNull(name)?.value?.invoke(
+            scope,
+            this,
+            args)
             ?: onNotFoundResult
             ?: scope.raiseSymbolNotFound(name)
 
@@ -251,6 +255,14 @@ open class Obj {
         scope.raiseNotImplemented()
     }
 
+    fun autoInstanceScope(parent: Scope): Scope  {
+       val scope = parent.copy(newThisObj = this, args = parent.args)
+        for( m in objClass.members) {
+            scope.objects[m.key] = m.value
+        }
+        return scope
+    }
+
     companion object {
 
         val rootObjectType = ObjClass("Obj").apply {

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjClass.kt

@@ -36,7 +36,7 @@ open class ObjClass(
     /**
      * members: fields most often. These are called with [ObjInstance] withs ths [ObjInstance.objClass]
      */
-    private val members = mutableMapOf<String, ObjRecord>()
+    internal val members = mutableMapOf<String, ObjRecord>()
 
     override fun toString(): String = className
 

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjFlow.kt

@@ -0,0 +1,136 @@
+package net.sergeych.lyng.obj
+
+import kotlinx.coroutines.DelicateCoroutinesApi
+import kotlinx.coroutines.channels.Channel
+import kotlinx.coroutines.channels.ChannelResult
+import kotlinx.coroutines.channels.ReceiveChannel
+import kotlinx.coroutines.channels.SendChannel
+import kotlinx.coroutines.sync.Mutex
+import kotlinx.coroutines.sync.withLock
+import net.sergeych.lyng.*
+import net.sergeych.mp_tools.globalLaunch
+import kotlin.coroutines.cancellation.CancellationException
+
+
+class ObjFlowBuilder(val output: SendChannel<Obj>) : Obj() {
+
+    override val objClass = type
+
+    companion object {
+        @OptIn(DelicateCoroutinesApi::class)
+        val type = object : ObjClass("FlowBuilder") {}.apply {
+            addFn("emit") {
+                val data = requireOnlyArg<Obj>()
+                try {
+                    val channel = thisAs<ObjFlowBuilder>().output
+                    if( !channel.isClosedForSend )
+                        channel.send(data)
+                    else
+                        throw ScriptFlowIsNoMoreCollected()
+                } catch (x: Exception) {
+                    if( x !is CancellationException )
+                        x.printStackTrace()
+                    throw ScriptFlowIsNoMoreCollected()
+                }
+                ObjVoid
+            }
+        }
+    }
+}
+
+private fun createLyngFlowInput(scope: Scope, producer: Statement): ReceiveChannel<Obj> {
+    val channel = Channel<Obj>(Channel.RENDEZVOUS)
+    val builder = ObjFlowBuilder(channel)
+    val builderScope = scope.copy(newThisObj = builder)
+    globalLaunch {
+        try {
+            producer.execute(builderScope)
+        }
+        catch(x: ScriptFlowIsNoMoreCollected) {
+            x.printStackTrace()
+            // premature flow closing, OK
+        }
+        catch(x: Exception) {
+            x.printStackTrace()
+        }
+        channel.close()
+    }
+    return channel
+}
+
+class ObjFlow(val producer: Statement, val scope: Scope) : Obj() {
+
+    override val objClass = type
+
+    companion object {
+        val type = object : ObjClass("Flow", ObjIterable) {
+            override suspend fun callOn(scope: Scope): Obj {
+                scope.raiseError("Flow constructor is not available")
+            }
+        }.apply {
+            addFn("iterator") {
+                val objFlow = thisAs<ObjFlow>()
+                ObjFlowIterator( statement { objFlow.producer.execute(ClosureScope(this,objFlow.scope)) } )
+            }
+        }
+    }
+}
+
+
+class ObjFlowIterator(val producer: Statement) : Obj() {
+
+    override val objClass: ObjClass = type
+
+    private var channel: ReceiveChannel<Obj>? = null
+
+    private var nextItem: ChannelResult<Obj>? = null
+
+    private var isCancelled = false
+
+    private fun checkNotCancelled(scope: Scope) {
+        if( isCancelled )
+            scope.raiseIllegalState("iteration is cancelled")
+    }
+    suspend fun hasNext(scope: Scope): ObjBool {
+        checkNotCancelled(scope)
+        // cold start:
+        if (channel == null) channel = createLyngFlowInput(scope, producer)
+        if (nextItem == null) nextItem = channel!!.receiveCatching()
+        return ObjBool(nextItem!!.isSuccess)
+    }
+
+    suspend fun next(scope: Scope): Obj {
+        checkNotCancelled(scope)
+        if (hasNext(scope).value == false) scope.raiseIllegalState("iteration is done")
+        return nextItem!!.getOrThrow().also { nextItem = null }
+    }
+
+    private val access = Mutex()
+    suspend fun cancel() {
+        access.withLock {
+            if (!isCancelled) {
+                isCancelled = true
+                channel?.cancel()
+            }
+        }
+    }
+
+    companion object {
+        val type = object : ObjClass("FlowIterator", ObjIterator) {
+
+        }.apply {
+            addFn("hasNext") {
+                thisAs<ObjFlowIterator>().hasNext(this).toObj()
+            }
+            addFn("next") {
+                val x = thisAs<ObjFlowIterator>()
+                x.next(this)
+            }
+            addFn("cancelIteration") {
+                val x = thisAs<ObjFlowIterator>()
+                x.cancel()
+                ObjVoid
+            }
+        }
+    }
+}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInstance.kt

@@ -34,7 +34,10 @@ class ObjInstance(override val objClass: ObjClass) : Obj() {
                                               onNotFoundResult: Obj?): Obj =
         instanceScope[name]?.let {
             if (it.visibility.isPublic)
-                it.value.invoke(scope, this, args)
+                it.value.invoke(
+                    instanceScope,
+                    this,
+                    args)
             else
                 scope.raiseError(ObjAccessException(scope, "can't invoke non-public method $name"))
         }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjIterable.kt

@@ -53,7 +53,7 @@ val ObjIterable by lazy {
         addFn("toMap") {
             val result = ObjMap()
             thisObj.toFlow(this).collect { pair ->
-                result.map[pair.getAt(this,0)] = pair.getAt(this, 1)
+                result.map[pair.getAt(this, 0)] = pair.getAt(this, 1)
             }
             result
         }
@@ -86,6 +86,24 @@ val ObjIterable by lazy {
             ObjList(result)
         }
 
+        addFn("take") {
+            var n = requireOnlyArg<ObjInt>().value.toInt()
+            val result = mutableListOf<Obj>()
+            if (n > 0) {
+                thisObj.enumerate(this) {
+                    result += it
+                    --n > 0
+                }
+            }
+            ObjList(result)
+        }
+
+//        addFn("drop" ) {
+//            var n = requireOnlyArg<ObjInt>().value.toInt()
+//            if( n < 0 ) raiseIllegalArgument("drop($n): should be positive")
+//            val it = callMethod<>()
+//        }
+
         addFn("isEmpty") {
             ObjBool(
                 thisObj.invokeInstanceMethod(this, "iterator")
@@ -93,6 +111,5 @@ val ObjIterable by lazy {
                     .not()
             )
         }
-
     }
 }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjIterator.kt

@@ -1,3 +1,4 @@
 package net.sergeych.lyng.obj
 
 val ObjIterator by lazy { ObjClass("Iterator") }
+

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjKotlinIterator.kt

@@ -36,7 +36,9 @@ class ObjKotlinObjIterator(val iterator: Iterator<Obj>) : Obj() {
             addFn("next") {
                 thisAs<ObjKotlinObjIterator>().iterator.next()
             }
-            addFn("hasNext") { thisAs<ObjKotlinIterator>().iterator.hasNext().toObj() }
+            addFn("hasNext") {
+                thisAs<ObjKotlinObjIterator>().iterator.hasNext().toObj()
+            }
         }
 
     }
@@ -56,3 +58,25 @@ fun Obj.toFlow(scope: Scope): Flow<Obj> = flow {
         emit(next.invoke(scope, iterator))
     }
 }
+
+/**
+ * Call [callback] for each element of this obj considering it provides [Iterator]
+ * methods `hasNext` and `next`.
+ *
+ * IF callback returns false, iteration is stopped.
+ */
+suspend fun Obj.enumerate(scope: Scope,callback: suspend (Obj)->Boolean) {
+    val iterator = invokeInstanceMethod(scope, "iterator")
+    val hasNext = iterator.getInstanceMethod(scope, "hasNext")
+    val next = iterator.getInstanceMethod(scope, "next")
+    var closeIt = false
+    while (hasNext.invoke(scope, iterator).toBool()) {
+        val nextValue = next.invoke(scope, iterator)
+        if( !callback(nextValue) ) {
+            closeIt = true
+            break
+        }
+    }
+    if( closeIt )
+        iterator.invokeInstanceMethod(scope, "cancelIteration", onNotFoundResult = ObjVoid)
+}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjRingBuffer.kt

@@ -0,0 +1,91 @@
+package net.sergeych.lyng.obj
+
+import net.sergeych.lyng.Scope
+
+class RingBuffer<T>(val maxSize: Int) : Iterable<T> {
+    private val data = arrayOfNulls<Any>(maxSize)
+
+    var size = 0
+        private set
+
+    private var start = 0
+
+    init {
+        check(maxSize > 0) { "Max size should be a positive number: $maxSize" }
+    }
+
+    fun add(item: T) {
+        if (size < maxSize)
+            size++
+        else
+            start = (start + 1) % maxSize
+        data[(start + size - 1) % maxSize] = item
+    }
+
+    @Suppress("unused")
+    fun addAll(vararg items: T) {
+        for (i in items) add(i)
+    }
+
+    @Suppress("unused")
+    fun addAll(elements: Iterable<T>) {
+        elements.forEach { add(it) }
+    }
+
+    @Suppress("unused")
+    fun clear() {
+        start = 0
+        size = 0
+        for (i in data.indices) {
+            data[i] = null
+        }
+    }
+
+    override fun iterator(): Iterator<T> =
+        object : Iterator<T> {
+            private var i = 0
+
+            override fun hasNext(): Boolean = i < size
+
+            override fun next(): T {
+                if (!hasNext()) throw NoSuchElementException()
+
+                @Suppress("UNCHECKED_CAST")
+                return data[(start + i++) % maxSize] as T
+            }
+        }
+}
+
+
+class ObjRingBuffer(val capacity: Int) : Obj() {
+    val buffer = RingBuffer<Obj>(capacity)
+
+    override val objClass: ObjClass = type
+
+    override suspend fun plusAssign(scope: Scope, other: Obj): Obj {
+        buffer.add(other.byValueCopy())
+        return this
+    }
+
+    companion object {
+        val type = object : ObjClass("RingBuffer", ObjIterable) {
+            override suspend fun callOn(scope: Scope): Obj {
+                return ObjRingBuffer(scope.requireOnlyArg<ObjInt>().toInt())
+            }
+        }.apply {
+            addFn("capacity") {
+                thisAs<ObjRingBuffer>().capacity.toObj()
+            }
+            addFn("size") {
+                thisAs<ObjRingBuffer>().buffer.size.toObj()
+            }
+            addFn("iterator") {
+                val buffer = thisAs<ObjRingBuffer>().buffer
+                ObjKotlinObjIterator(buffer.iterator())
+            }
+            addFn("add") {
+                thisAs<ObjRingBuffer>().apply { buffer.add(requireOnlyArg<Obj>()) }
+            }
+        }
+    }
+}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjString.kt

@@ -93,7 +93,9 @@ data class ObjString(val value: String) : Obj() {
                 ObjString(decoder.unpackBinaryData().decodeToString())
         }.apply {
             addFn("toInt") {
-                ObjInt(thisAs<ObjString>().value.toLong())
+                ObjInt(thisAs<ObjString>().value.toLongOrNull()
+                    ?: raiseIllegalArgument("can't convert to int: $thisObj")
+                )
             }
             addFn("startsWith") {
                 ObjBool(thisAs<ObjString>().value.startsWith(requiredArg<ObjString>(0).value))
@@ -137,7 +139,9 @@ data class ObjString(val value: String) : Obj() {
             }
             addFn("encodeUtf8") { ObjBuffer(thisAs<ObjString>().value.encodeToByteArray().asUByteArray()) }
             addFn("size") { ObjInt(thisAs<ObjString>().value.length.toLong()) }
-            addFn("toReal") { ObjReal(thisAs<ObjString>().value.toDouble()) }
+            addFn("toReal") {
+                ObjReal(thisAs<ObjString>().value.toDouble())
+            }
         }
     }
 }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/pacman/ImportProvider.kt

@@ -1,6 +1,7 @@
 package net.sergeych.lyng.pacman
 
 import net.sergeych.lyng.*
+import net.sergeych.mptools.CachedExpression
 
 /**
  * Package manager INTERFACE (abstract class). Performs import routines
@@ -45,6 +46,15 @@ abstract class ImportProvider(
 
     fun newModuleAt(pos: Pos): ModuleScope =
         ModuleScope(this, pos, "unknown")
+
+    private var cachedStdScope = CachedExpression<Scope>()
+
+    suspend fun newStdScope(pos: Pos = Pos.builtIn): Scope =
+        cachedStdScope.get {
+            newModuleAt(pos).also {
+                it.eval("import lyng.stdlib\n")
+            }
+        }.copy()
 }
 
 

lynglib/src/commonMain/kotlin/net/sergeych/lyng/stdlib_included/root_lyng.kt

@@ -0,0 +1,58 @@
+package net.sergeych.lyng.stdlib_included
+internal val rootLyng = """
+package lyng.stdlib
+    
+fun Iterable.filter(predicate) {
+    val list = this
+    flow {
+        for( item in list ) {
+            if( predicate(item) ) {
+                emit(item)
+            }
+        }
+    }
+}
+
+fun Iterable.drop(n) {
+    var cnt = 0
+    filter { cnt++ >= n }
+}
+
+fun Iterable.first() {
+    val i = iterator()
+    if( !i.hasNext() ) throw NoSuchElementException()
+    i.next()
+}
+
+fun Iterable.last() {
+    var found = false
+    var element = null
+    for( i in this ) {
+        element = i
+        found = true
+    }
+    if( !found ) throw NoSuchElementException()
+    element
+}
+
+fun Iterable.dropLast(n) {
+    val list = this
+    val buffer = RingBuffer(n)
+    flow {
+        for( item in list ) {
+            if( buffer.size == n ) 
+                emit( buffer.first() )
+            buffer += item
+        }
+    }
+}
+
+fun Iterable.takeLast(n) {
+    val list = this
+    val buffer = RingBuffer(n)
+    for( item in list ) buffer += item
+    buffer
+}
+    
+""".trimIndent()
+

lynglib/src/commonMain/lyng/stdlib/Iterable.lyng

@@ -0,0 +1,16 @@
+
+fun Iterable.filter( predicate ) {
+    flow {
+        for( item in this )
+            if( predicate(item) )
+                emit(item)
+    }
+}
+
+fun Iterable.drop(n) {
+    require( n >= 0, "drop amount must be non-negative")
+    var count = 0
+    filter {
+        count++ < N
+    }
+}

lynglib/src/commonTest/kotlin/CoroutinesTest.kt

@@ -29,7 +29,7 @@ class TestCoroutines {
             val done = CompletableDeferred()
             
             launch { 
-                delay(10)
+                delay(30)
                 done.complete("ok")
             }
             
@@ -63,4 +63,109 @@ class TestCoroutines {
     """.trimIndent()
         )
     }
+
+    @Test
+    fun testFlows() = runTest {
+        eval("""
+            val f = flow {
+                println("Starting generator")
+                var n1 = 0
+                var n2 = 1
+                emit(n1)
+                emit(n2)
+                while(true) {
+                    val n = n1 + n2
+                    emit(n)
+                    n1 = n2
+                    n2 = n
+                }
+            }
+            val correctFibs = [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, 144, 233, 377, 610, 987, 1597, 2584, 4181, 6765]
+            assertEquals( correctFibs, f.take(correctFibs.size))
+            
+        """.trimIndent())
+    }
+
+    @Test
+    fun testFlow2() = runTest {
+        eval("""
+                val f = flow {
+                        emit("start")
+                        emit("start2")
+                        (1..4).forEach { 
+                            emit(it) 
+                        }
+                }
+                // let's collect flow:
+                val result = []
+                for( x in f ) result += x
+                    println(result)
+                
+                // let's collect it once again:
+                assertEquals( result, f.toList())
+                assertEquals( result, f.toList())
+        """.trimIndent())
+    }
+
+    @Test
+    fun testFlowClosures() = runTest {
+        eval("""
+            fun filter( a, b ) {
+                println("filter: %s, %s"(a,b))
+                flow {
+                    emit(a)
+                    emit(b)
+                }
+            }
+            
+            assertEquals( [5, 1], filter(5,1).toList() )
+            assertEquals( [2, 3], filter(2,3).toList() )
+            
+        """.trimIndent())
+    }
+
+    @Test
+    fun testFilterFlow() = runTest {
+        eval("""
+            fun filter( list, predicate ) {
+                val p = predicate
+                println("predicate "+predicate+" / "+p)
+                flow {
+                    // here p is captured only once and does not change!
+                    for( item in list ) {
+                        print("filter "+p+" "+item+": ")
+                        if( p(item) ) {
+                            println("OK")
+                            emit(item)
+                        }
+                        else println("NO")
+                    }
+                }
+            }
+            
+//            fun drop(i, n) {
+//                require( n >= 0, "drop amount must be non-negative")
+//                var count = 0
+//                println("drop %d"(n))
+//                filter(i) {
+//                    count++ >= n
+//                }
+//            }
+            
+            val src = (1..1).toList()
+            assertEquals( 1, filter(src) { true }.toList().size )
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            println("----------------------------------------------------------")
+            assertEquals( 0, filter(src) { false }.toList().size )
+//            assertEquals( 3, filter(src) { true }.size() ) 
+            
+//            assertEquals( [7,8], drop((1..8).toList(),6).toList())
+//            assertEquals( [1,3,5,7], filter((1..8).toList()) { 
+//                println("call2")
+//                it % 2 == 1 
+//            }.toList())
+        """.trimIndent())
+    }
 }

lynglib/src/commonTest/kotlin/ScriptTest.kt

@@ -1274,6 +1274,106 @@ class ScriptTest {
         )
     }
 
+    @Test
+    fun testCaptureLocals() = runTest {
+        eval(
+            """
+            
+            fun outer(prefix) {
+                val p1 = "0" + prefix
+                { 
+                    p1 + "2" + it
+                }
+            }
+            fun outer2(prefix) {
+                val p1 = "*" + prefix
+                { 
+                    p1 + "2" + it
+                }
+            }
+            val x = outer("1")
+            val y = outer2("1")
+            println(x("!"))
+            assertEquals( "0123", x("3") )
+            assertEquals( "*123", y("3") )
+        """.trimIndent()
+        )
+    }
+
+    @Test
+    fun testInstanceCallScopeIsCorrect() = runTest {
+        eval(
+            """
+            
+            val prefix = ":"
+            
+            class T(text) {
+                fun getText() { 
+                    println(text)
+                    prefix + text + "!" 
+                }
+            }
+            
+            val text = "invalid"
+            
+            val t1 = T("foo")
+            val t2 = T("bar")
+            
+            // get inside the block
+            for( i in 1..3 ) {
+                assertEquals( "foo", t1.text )
+                assertEquals( ":foo!", t1.getText() )
+                assertEquals( "bar", t2.text )
+                assertEquals( ":bar!", t2.getText() )
+            }
+        """.trimIndent()
+        )
+    }
+
+    @Test
+    fun testAppliedScopes() = runTest {
+        eval(
+            """
+            class T(text) {
+                fun getText() { 
+                    println(text)
+                    text + "!" 
+                }
+            }
+        
+            val prefix = ":"
+            val lambda = { 
+                prefix + getText() + "!"
+            }
+        
+            val text = "invalid"
+            val t1 = T("foo")
+            val t2 = T("bar")
+            
+            t1.apply { 
+                // it must take "text" from class t1:
+                assertEquals("foo", text)
+                assertEquals( "foo!", getText() ) 
+                assertEquals( ":foo!!", lambda() ) 
+            } 
+            t2.apply { 
+                assertEquals("bar", text)
+                assertEquals( "bar!", getText() ) 
+                assertEquals( ":bar!!", lambda() ) 
+            }
+            // worst case: names clash
+            fun badOne() {
+                val prefix = "&"
+                t1.apply {
+                    assertEquals( ":foo!!", lambda() ) 
+                }
+            }
+            badOne()
+            
+            """.trimIndent()
+        )
+    }
+
     @Test
     fun testLambdaWithArgsEllipsis() = runTest {
         eval(
@@ -2244,10 +2344,12 @@ class ScriptTest {
 
     @Test
     fun testSet2() = runTest {
-        eval("""
+        eval(
+            """
             assertEquals( Set( ...[1,2,3]), Set(1,2,3) )
             assertEquals( Set( ...[1,false,"ok"]), Set("ok", 1, false) )
-        """.trimIndent())
+        """.trimIndent()
+        )
     }
 
     @Test
@@ -2284,9 +2386,11 @@ class ScriptTest {
         eval(
             """
             class Point(x,y)
+            
             // see the difference: apply changes this to newly created Point:
             val p = Point(1,2).apply { 
-                this.x++; this.y++ 
+                this.x++ 
+                y++ 
             }
             assertEquals(p, Point(2,3))
             >>> void
@@ -2305,6 +2409,10 @@ class ScriptTest {
             }
             
             fun Object.isInteger() {
+                println(this)
+                println(this is Int)
+                println(this is Real)
+                println(this is String)
                 when(this) {
                     is Int -> true
                     is Real -> toInt() == this
@@ -2319,7 +2427,7 @@ class ScriptTest {
             assert( 12.isInteger() == true )
             assert( 12.1.isInteger() == false )
             assert( "5".isInteger() )
-            assert( ! "5.2".isInteger() )
+            assert( !"5.2".isInteger() )
         """.trimIndent()
         )
     }
@@ -2446,20 +2554,26 @@ class ScriptTest {
     fun testDefaultImportManager() = runTest {
         val scope = Scope.new()
         assertFails {
-            scope.eval("""
+            scope.eval(
+                """
                 import foo
                 foo()
-                """.trimIndent())
+                """.trimIndent()
+            )
         }
-        scope.importManager.addTextPackages("""
+        scope.importManager.addTextPackages(
+            """
             package foo
             
             fun foo() { "bar" }
-        """.trimIndent())
+        """.trimIndent()
-        scope.eval("""
+        )
+        scope.eval(
+            """
                 import foo
                 assertEquals( "bar", foo())
-                """.trimIndent())
+                """.trimIndent()
+        )
     }
 
     @Test
@@ -2478,7 +2592,8 @@ class ScriptTest {
 
     @Test
     fun testBuffer() = runTest {
-        eval("""
+        eval(
+            """
             import lyng.buffer
             
             assertEquals( 0, Buffer().size )
@@ -2497,12 +2612,14 @@ class ScriptTest {
             assertEquals(101, buffer[2])
             assertEquals("Heelo", buffer.decodeUtf8())
             
-        """.trimIndent())
+        """.trimIndent()
+        )
     }
 
     @Test
     fun testBufferCompare() = runTest {
-        eval("""
+        eval(
+            """
             import lyng.buffer
             
             println("Hello".characters())
@@ -2520,7 +2637,8 @@ class ScriptTest {
             assertEquals("foo",  map[b2])
             assertEquals(null,  map[b3])
             
-        """.trimIndent())
+        """.trimIndent()
+        )
     }
 
     @Test
@@ -2548,6 +2666,7 @@ class ScriptTest {
         )
         delay(1000)
     }
+
     @Test
     fun testTimeStatics() = runTest {
         eval(
@@ -2589,7 +2708,8 @@ class ScriptTest {
         println(Script.defaultImportManager.packageNames)
         println(s.importManager.packageNames)
 
-        s.importManager.addTextPackages("""
+        s.importManager.addTextPackages(
+            """
             package foo
              
             import lyng.time
@@ -2597,8 +2717,10 @@ class ScriptTest {
             fun foo() {
                 println("foo: %s"(Instant()))
             }
-        """.trimIndent())
+        """.trimIndent()
-        s.importManager.addTextPackages("""
+        )
+        s.importManager.addTextPackages(
+            """
             package bar
              
             import lyng.time
@@ -2606,24 +2728,28 @@ class ScriptTest {
             fun bar() {
                 println("bar: %s"(Instant()))
             }
-        """.trimIndent())
+        """.trimIndent()
+        )
 
         println(s.importManager.packageNames)
 
-        s.eval("""
+        s.eval(
+            """
             import foo
             import bar
             
             foo()
             bar()
             
-        """.trimIndent())
+        """.trimIndent()
+        )
 
     }
 
     @Test
     fun testIndexIntIncrements() = runTest {
-        eval("""
+        eval(
+            """
         val x = [1,2,3]
         x[1]++
         ++x[0]
@@ -2636,12 +2762,14 @@ class ScriptTest {
         assert( b == Buffer(1,3,3) )
         ++b[0]
         assertEquals( b, Buffer(2,3,3) )
-        """.trimIndent())
+        """.trimIndent()
+        )
     }
 
     @Test
     fun testIndexIntDecrements() = runTest {
-        eval("""
+        eval(
+            """
         val x = [1,2,3]
         x[1]--
         --x[0]
@@ -2654,13 +2782,14 @@ class ScriptTest {
         assert( b == Buffer(1,1,3) )
         --b[0]
         assertEquals( b, Buffer(0,1,3) )
-        """.trimIndent())
+        """.trimIndent()
+        )
     }
 
     @Test
     fun testRangeToList() = runTest {
         val x = eval("""(1..10).toList()""") as ObjList
-        assertEquals(listOf(1,2,3,4,5,6,7,8,9,10), x.list.map { it.toInt() })
+        assertEquals(listOf(1, 2, 3, 4, 5, 6, 7, 8, 9, 10), x.list.map { it.toInt() })
         val y = eval("""(-2..3).toList()""") as ObjList
         println(y.list)
     }

lynglib/src/commonTest/kotlin/StdlibTest.kt

@@ -0,0 +1,65 @@
+import kotlinx.coroutines.test.runTest
+import net.sergeych.lyng.eval
+import kotlin.test.Test
+
+class StdlibTest {
+    @Test
+    fun testIterableFilter() = runTest {
+        eval("""
+            assertEquals([1,3,5,7], (1..8).filter{ it % 2 == 1 }.toList() )
+            assertEquals([2,4,6,8], (1..8).filter{ it % 2 == 0 }.toList() )
+        """.trimIndent())
+
+    }
+
+    @Test
+    fun testFirstLast() = runTest {
+        eval("""
+            assertEquals(1, (1..8).first )
+            assertEquals(8, (1..8).last )
+        """.trimIndent())
+    }
+
+    @Test
+    fun testTake() = runTest {
+        eval("""
+            assertEquals([1,2,3], (1..8).take(3).toList() )
+            assertEquals([7,8], (1..8).takeLast(2).toList() )
+        """.trimIndent())
+    }
+
+    @Test
+    fun testRingBuffer() = runTest {
+        eval("""
+            val r = RingBuffer(3)
+            assert( r is RingBuffer )
+            assertEquals(0, r.size)
+            assertEquals(3, r.capacity)
+            
+            r += 10
+            assertEquals(1, r.size)
+            assertEquals(10, r.first)
+            
+            r += 20
+            assertEquals(2, r.size)
+            assertEquals( [10, 20], r.toList() )
+            
+            r += 30
+            assertEquals(3, r.size)
+            assertEquals( [10, 20, 30], r.toList() )
+            
+            r += 40
+            assertEquals(3, r.size)
+            assertEquals( [20, 30, 40], r.toList() )
+            
+        """.trimIndent())
+    }
+
+    @Test
+    fun testDrop() = runTest {
+        eval("""
+            assertEquals([7,8], (1..8).drop(6).toList() )
+            assertEquals([1,2], (1..8).dropLast(6).toList() )
+        """.trimIndent())
+    }
+}

lynglib/src/jvmTest/kotlin/BookTest.kt

@@ -5,6 +5,7 @@ import kotlinx.coroutines.flow.flowOn
 import kotlinx.coroutines.runBlocking
 import kotlinx.coroutines.test.runTest
 import net.sergeych.lyng.Scope
+import net.sergeych.lyng.Script
 import net.sergeych.lyng.obj.ObjVoid
 import java.nio.file.Files
 import java.nio.file.Files.readAllLines
@@ -163,9 +164,10 @@ fun parseDocTests(fileName: String, bookMode: Boolean = false): Flow<DocTest> =
 }
     .flowOn(Dispatchers.IO)
 
-suspend fun DocTest.test(scope: Scope = Scope()) {
+suspend fun DocTest.test(_scope: Scope? = null) {
     val collectedOutput = StringBuilder()
     val currentTest = this
+    val scope  = _scope ?: Script.newScope()
     scope.apply {
         addFn("println") {
             if( bookMode ) {
@@ -304,4 +306,9 @@ class BookTest {
         runDocTests("../docs/parallelism.md")
     }
 
+    @Test
+    fun testRingBuffer() = runBlocking {
+        runDocTests("../docs/RingBuffer.md")
+    }
+
 }