0.8.4: stadlib in Lyng! added universal Iterable functions

docs/Iterable.md

@@ -17,7 +17,14 @@ Iterator itself is a simple interface that should provide only to method:
 
 Just remember at this stage typed declarations are not yet supported.
 
-Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions available:
+Having `Iterable` in base classes allows to use it in for loop. Also, each `Iterable` has some utility functions available, for example
+
+    val r = 1..10  // Range is Iterable!  
+    assertEquals( [9,10] r.takeLast(2) )
+    assertEquals( [1,2,3] r.take(3) )
+    assertEquals( [9,10] r.drop(8) )
+    assertEquals( [1,2] r.dropLast(8) )
+    >>> void
 
 ## Instance methods
 
@@ -33,7 +40,15 @@ Having `Iterable` in base classes allows to use it in for loop. Also, each `Iter
 | map(f)          | create a list of values returned by `f` called for each element of the iterable |
 | indexOf(i)      | return index if the first encounter of i or a negative value if not found       |
 | associateBy(kf) | create a map where keys are returned by kf that will be called for each element |
+| first           | first element (1)                                                               |
+| last            | last element (1)                                                                |
+| take(n)         | return [Iterable] of up to n first elements                                     |
+| taleLast(n)     | return [Iterable] of up to n last elements                                      |
+| drop(n)         | return new [Iterable] without first n elements                                  |
+| dropLast(n)     | return new [Iterable] without last n elements                                   |
 
+(1)
+: throws `NoSuchElementException` if there is no such element 
 
     fun Iterable.toList(): List
     fun Iterable.toSet(): Set

docs/RingBuffer.md

@@ -0,0 +1,52 @@
+# RingBuffer
+
+This is a fixed size buffer that allow to store N last elements with _O(1)_ effectiveness (no data shifting).
+
+Here is the sample:
+
+    val r = RingBuffer(3)
+    assert( r is RingBuffer )
+    assertEquals(0, r.size)
+    assertEquals(3, r.capacity)
+    
+    r += 10
+    assertEquals(1, r.size)
+    assertEquals(10, r.first)
+    
+    r += 20
+    assertEquals(2, r.size)
+    assertEquals( [10, 20], r.toList() )
+    
+    r += 30
+    assertEquals(3, r.size)
+    assertEquals( [10, 20, 30], r.toList() )
+    
+    // now first value is lost:
+    r += 40
+    assertEquals(3, r.size)
+    assertEquals( [20, 30, 40], r.toList() )
+    assertEquals(3, r.capacity)
+
+    >>> void
+
+Ring buffer implements [Iterable], so any of its methods are available for `RingBuffer`, e.g. `first`, `last`, `toList`,
+`take`, `drop`, `takelast`, `dropLast`, etc.
+
+## Constructor
+
+    RinbBuffer(capacity: Int)
+
+## Instance methods
+
+| method      | description            | remarks |
+|-------------|------------------------|---------|
+| capacity    | max size of the buffer |         |
+| size        | current size           | (1)     |
+| operator += | add new item           | (1)     |
+| add(item)   | add new item           | (1)     |
+| iterator()  | return iterator        | (1)     |
+
+(1)
+: Ringbuffer is not threadsafe, protect it with a mutex to avoid RC where necessary.
+
+[Iterable]: Iterable.md

docs/parallelism.md

@@ -189,3 +189,19 @@ Important difference from the channels or like, every time you collect the flow,
 
 Notice that flow's lambda is not called until actual collection is started. Cold flows are
 better in terms of resource consumption.
+
+Flows allow easy transforming of any [Iterable]. See how the standard Lyng library functions use it:
+
+    fun Iterable.filter(predicate) {
+        val list = this
+        flow {
+            for( item in list ) {
+                if( predicate(item) ) {
+                    emit(item)
+                }
+            }
+        }
+    }
+
+
+[Iterable]: Iterable.md

docs/tutorial.md

@@ -14,8 +14,9 @@ __Other documents to read__ maybe after this one:
 - [Advanced topics](advanced_topics.md), [declaring arguments](declaring_arguments.md)
 - [OOP notes](OOP.md), [exception handling](exceptions_handling.md)
 - [math in Lyng](math.md)
+- [time](time.md) and [parallelism](parallelism.md)
 - [parallelism] - multithreaded code, coroutines, etc.
-- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md)
+- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md), [RingBuffer], [Buffer].
 - Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and
   loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
 
@@ -1305,3 +1306,5 @@ See [math functions](math.md). Other general purpose functions are:
 [Buffer]: Buffer.md
 
 [parallelism]: parallelism.md
+
+[RingBuffer]: RingBuffer.md

lynglib/build.gradle.kts

@@ -4,7 +4,7 @@ import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
 import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 
 group = "net.sergeych"
-version = "0.8.3-SNAPSHOT"
+version = "0.8.4-SNAPSHOT"
 
 buildscript {
     repositories {

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Scope.kt

@@ -163,13 +163,14 @@ open class Scope(
     fun addConst(name: String, value: Obj) = addItem(name, false, value)
 
     suspend fun eval(code: String): Obj =
-        Compiler.compile(code.toSource(), currentImportProvider).execute(this)
+        eval(code.toSource())
 
-    suspend fun eval(source: Source): Obj =
+    suspend fun eval(source: Source): Obj {
-        Compiler.compile(
+        return Compiler.compile(
             source,
             currentImportProvider
         ).execute(this)
+    }
 
     fun containsLocal(name: String): Boolean = name in objects
 

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Script.kt

@@ -4,6 +4,7 @@ import kotlinx.coroutines.delay
 import kotlinx.coroutines.yield
 import net.sergeych.lyng.obj.*
 import net.sergeych.lyng.pacman.ImportManager
+import net.sergeych.lyng.stdlib_included.rootLyng
 import net.sergeych.lynon.ObjLynonClass
 import net.sergeych.mp_tools.globalDefer
 import kotlin.math.*
@@ -22,10 +23,14 @@ class Script(
         return lastResult
     }
 
-    suspend fun execute() = execute(defaultImportManager.newModule())
+    suspend fun execute() = execute(
+        defaultImportManager.newStdScope()
+    )
 
     companion object {
 
+        suspend fun newScope(pos: Pos = Pos.builtIn) = defaultImportManager.newStdScope(pos)
+
         internal val rootScope: Scope = Scope(null).apply {
             ObjException.addExceptionsToContext(this)
             addFn("print") {
@@ -202,6 +207,7 @@ class Script(
             addConst("Iterable", ObjIterable)
             addConst("Collection", ObjCollection)
             addConst("Array", ObjArray)
+            addConst("RingBuffer", ObjRingBuffer.type)
             addConst("Class", ObjClassType)
 
             addConst("Deferred", ObjDeferred.type)
@@ -235,6 +241,9 @@ class Script(
 
         val defaultImportManager: ImportManager by lazy {
             ImportManager(rootScope, SecurityManager.allowAll).apply {
+                addTextPackages(
+                    rootLyng
+                )
                 addPackage("lyng.buffer") {
                     it.addConst("Buffer", ObjBuffer.type)
                     it.addConst("MutableBuffer", ObjMutableBuffer.type)

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjKotlinIterator.kt

@@ -36,7 +36,9 @@ class ObjKotlinObjIterator(val iterator: Iterator<Obj>) : Obj() {
             addFn("next") {
                 thisAs<ObjKotlinObjIterator>().iterator.next()
             }
-            addFn("hasNext") { thisAs<ObjKotlinIterator>().iterator.hasNext().toObj() }
+            addFn("hasNext") {
+                thisAs<ObjKotlinObjIterator>().iterator.hasNext().toObj()
+            }
         }
 
     }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjRingBuffer.kt

@@ -0,0 +1,91 @@
+package net.sergeych.lyng.obj
+
+import net.sergeych.lyng.Scope
+
+class RingBuffer<T>(val maxSize: Int) : Iterable<T> {
+    private val data = arrayOfNulls<Any>(maxSize)
+
+    var size = 0
+        private set
+
+    private var start = 0
+
+    init {
+        check(maxSize > 0) { "Max size should be a positive number: $maxSize" }
+    }
+
+    fun add(item: T) {
+        if (size < maxSize)
+            size++
+        else
+            start = (start + 1) % maxSize
+        data[(start + size - 1) % maxSize] = item
+    }
+
+    @Suppress("unused")
+    fun addAll(vararg items: T) {
+        for (i in items) add(i)
+    }
+
+    @Suppress("unused")
+    fun addAll(elements: Iterable<T>) {
+        elements.forEach { add(it) }
+    }
+
+    @Suppress("unused")
+    fun clear() {
+        start = 0
+        size = 0
+        for (i in data.indices) {
+            data[i] = null
+        }
+    }
+
+    override fun iterator(): Iterator<T> =
+        object : Iterator<T> {
+            private var i = 0
+
+            override fun hasNext(): Boolean = i < size
+
+            override fun next(): T {
+                if (!hasNext()) throw NoSuchElementException()
+
+                @Suppress("UNCHECKED_CAST")
+                return data[(start + i++) % maxSize] as T
+            }
+        }
+}
+
+
+class ObjRingBuffer(val capacity: Int) : Obj() {
+    val buffer = RingBuffer<Obj>(capacity)
+
+    override val objClass: ObjClass = type
+
+    override suspend fun plusAssign(scope: Scope, other: Obj): Obj {
+        buffer.add(other.byValueCopy())
+        return this
+    }
+
+    companion object {
+        val type = object : ObjClass("RingBuffer", ObjIterable) {
+            override suspend fun callOn(scope: Scope): Obj {
+                return ObjRingBuffer(scope.requireOnlyArg<ObjInt>().toInt())
+            }
+        }.apply {
+            addFn("capacity") {
+                thisAs<ObjRingBuffer>().capacity.toObj()
+            }
+            addFn("size") {
+                thisAs<ObjRingBuffer>().buffer.size.toObj()
+            }
+            addFn("iterator") {
+                val buffer = thisAs<ObjRingBuffer>().buffer
+                ObjKotlinObjIterator(buffer.iterator())
+            }
+            addFn("add") {
+                thisAs<ObjRingBuffer>().apply { buffer.add(requireOnlyArg<Obj>()) }
+            }
+        }
+    }
+}

lynglib/src/commonMain/kotlin/net/sergeych/lyng/pacman/ImportProvider.kt

@@ -1,6 +1,7 @@
 package net.sergeych.lyng.pacman
 
 import net.sergeych.lyng.*
+import net.sergeych.mptools.CachedExpression
 
 /**
  * Package manager INTERFACE (abstract class). Performs import routines
@@ -45,6 +46,15 @@ abstract class ImportProvider(
 
     fun newModuleAt(pos: Pos): ModuleScope =
         ModuleScope(this, pos, "unknown")
+
+    private var cachedStdScope = CachedExpression<Scope>()
+
+    suspend fun newStdScope(pos: Pos = Pos.builtIn): Scope =
+        cachedStdScope.get {
+            newModuleAt(pos).also {
+                it.eval("import lyng.stdlib\n")
+            }
+        }.copy()
 }
 
 

lynglib/src/commonMain/kotlin/net/sergeych/lyng/stdlib_included/root_lyng.kt

@@ -1,5 +1,58 @@
 package net.sergeych.lyng.stdlib_included
-
 internal val rootLyng = """
+package lyng.stdlib
+    
+fun Iterable.filter(predicate) {
+    val list = this
+    flow {
+        for( item in list ) {
+            if( predicate(item) ) {
+                emit(item)
+            }
+        }
+    }
+}
+
+fun Iterable.drop(n) {
+    var cnt = 0
+    filter { cnt++ >= n }
+}
+
+fun Iterable.first() {
+    val i = iterator()
+    if( !i.hasNext() ) throw NoSuchElementException()
+    i.next()
+}
+
+fun Iterable.last() {
+    var found = false
+    var element = null
+    for( i in this ) {
+        element = i
+        found = true
+    }
+    if( !found ) throw NoSuchElementException()
+    element
+}
+
+fun Iterable.dropLast(n) {
+    val list = this
+    val buffer = RingBuffer(n)
+    flow {
+        for( item in list ) {
+            if( buffer.size == n ) 
+                emit( buffer.first() )
+            buffer += item
+        }
+    }
+}
+
+fun Iterable.takeLast(n) {
+    val list = this
+    val buffer = RingBuffer(n)
+    for( item in list ) buffer += item
+    buffer
+}
     
 """.trimIndent()
+

lynglib/src/commonTest/kotlin/CoroutinesTest.kt

@@ -29,7 +29,7 @@ class TestCoroutines {
             val done = CompletableDeferred()
             
             launch { 
-                delay(10)
+                delay(30)
                 done.complete("ok")
             }
             
@@ -124,7 +124,6 @@ class TestCoroutines {
         """.trimIndent())
     }
 
-
     @Test
     fun testFilterFlow() = runTest {
         eval("""

lynglib/src/commonTest/kotlin/StdlibTest.kt

@@ -0,0 +1,65 @@
+import kotlinx.coroutines.test.runTest
+import net.sergeych.lyng.eval
+import kotlin.test.Test
+
+class StdlibTest {
+    @Test
+    fun testIterableFilter() = runTest {
+        eval("""
+            assertEquals([1,3,5,7], (1..8).filter{ it % 2 == 1 }.toList() )
+            assertEquals([2,4,6,8], (1..8).filter{ it % 2 == 0 }.toList() )
+        """.trimIndent())
+
+    }
+
+    @Test
+    fun testFirstLast() = runTest {
+        eval("""
+            assertEquals(1, (1..8).first )
+            assertEquals(8, (1..8).last )
+        """.trimIndent())
+    }
+
+    @Test
+    fun testTake() = runTest {
+        eval("""
+            assertEquals([1,2,3], (1..8).take(3).toList() )
+            assertEquals([7,8], (1..8).takeLast(2).toList() )
+        """.trimIndent())
+    }
+
+    @Test
+    fun testRingBuffer() = runTest {
+        eval("""
+            val r = RingBuffer(3)
+            assert( r is RingBuffer )
+            assertEquals(0, r.size)
+            assertEquals(3, r.capacity)
+            
+            r += 10
+            assertEquals(1, r.size)
+            assertEquals(10, r.first)
+            
+            r += 20
+            assertEquals(2, r.size)
+            assertEquals( [10, 20], r.toList() )
+            
+            r += 30
+            assertEquals(3, r.size)
+            assertEquals( [10, 20, 30], r.toList() )
+            
+            r += 40
+            assertEquals(3, r.size)
+            assertEquals( [20, 30, 40], r.toList() )
+            
+        """.trimIndent())
+    }
+
+    @Test
+    fun testDrop() = runTest {
+        eval("""
+            assertEquals([7,8], (1..8).drop(6).toList() )
+            assertEquals([1,2], (1..8).dropLast(6).toList() )
+        """.trimIndent())
+    }
+}

lynglib/src/jvmTest/kotlin/BookTest.kt

@@ -5,6 +5,7 @@ import kotlinx.coroutines.flow.flowOn
 import kotlinx.coroutines.runBlocking
 import kotlinx.coroutines.test.runTest
 import net.sergeych.lyng.Scope
+import net.sergeych.lyng.Script
 import net.sergeych.lyng.obj.ObjVoid
 import java.nio.file.Files
 import java.nio.file.Files.readAllLines
@@ -163,9 +164,10 @@ fun parseDocTests(fileName: String, bookMode: Boolean = false): Flow<DocTest> =
 }
     .flowOn(Dispatchers.IO)
 
-suspend fun DocTest.test(scope: Scope = Scope()) {
+suspend fun DocTest.test(_scope: Scope? = null) {
     val collectedOutput = StringBuilder()
     val currentTest = this
+    val scope  = _scope ?: Script.newScope()
     scope.apply {
         addFn("println") {
             if( bookMode ) {
@@ -304,4 +306,9 @@ class BookTest {
         runDocTests("../docs/parallelism.md")
     }
 
+    @Test
+    fun testRingBuffer() = runBlocking {
+        runDocTests("../docs/RingBuffer.md")
+    }
+
 }