ref #56 exceptions are serializable

+fixed ambigity betwee lyng/kotln toString, added directional parameter to kotlin

docs/exceptions_handling.md

@@ -122,6 +122,32 @@ This way, in turn, can also be shortened, as it is overly popular:
 The trick, though, works with strings only, and always provide `Exception` instances, which is good for debugging but
 most often not enough.
 
+# Exception class
+
+Serializable class that conveys information about the exception. Important members and methods are:
+
+| name              | description                                            |
+|-------------------|--------------------------------------------------------|
+| message           | String message                                         |
+| stackTrace        | lyng stack trace, list of `StackTraceEntry`, see below |
+| printStackTrace() | format and print stack trace using println()           |
+
+## StackTraceEntry
+
+A simple structire that stores single entry in Lyng stack, it is created automatically on exception creation:
+
+```kotlin
+class StackTraceEntry(
+    val sourceName: String,
+    val line: Int,
+    val column: Int,
+    val sourceString: String
+)
+```
+
+- `sourceString` is a line extracted from sources. Note that it _is serialized and printed_, so if you want to conceal it, catch all exceptions and filter out sensitive information.
+
+
 # Custom error classes
 
 _this functionality is not yet released_
@@ -131,7 +157,7 @@ _this functionality is not yet released_
 | class                      | notes                                                 |
 |----------------------------|-------------------------------------------------------|
 | Exception                  | root of al throwable objects                          |
-| NullReferenceException       |                                                       |
+| NullReferenceException     |                                                       |
 | AssertionFailedException   |                                                       | 
 | ClassCastException         |                                                       |
 | IndexOutOfBoundsException  |                                                       |

docs/tutorial.md

@@ -16,7 +16,8 @@ __Other documents to read__ maybe after this one:
 - [math in Lyng](math.md)
 - [time](time.md) and [parallelism](parallelism.md)
 - [parallelism] - multithreaded code, coroutines, etc.
-- Some class references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md), [RingBuffer], [Buffer].
+- Some class
+  references: [List], [Set], [Map], [Real], [Range], [Iterable], [Iterator], [time manipulation](time.md), [RingBuffer], [Buffer].
 - Some samples: [combinatorics](samples/combinatorics.lyng.md), national vars and
   loops: [сумма ряда](samples/сумма_ряда.lyng.md). More at [samples folder](samples)
 
@@ -488,7 +489,8 @@ Lyng has built-in mutable array class `List` with simple literals:
     [1, "two", 3.33].size
     >>> 3
 
-[List] is an implementation of the type `Array`, and through it `Collection` and [Iterable]. Please read [Iterable], many collection based methods are implemented there.
+[List] is an implementation of the type `Array`, and through it `Collection` and [Iterable]. Please read [Iterable],
+many collection based methods are implemented there.
 
 Lists can contain any type of objects, lists too:
 
@@ -1137,7 +1139,8 @@ See [more docs on time manipulation](time.md)
 
 # Enums
 
-For the moment, only simple enums are implemented. Enum is a list of constants, represented also by their _ordinal_ - [Int] value.
+For the moment, only simple enums are implemented. Enum is a list of constants, represented also by their
+_ordinal_ - [Int] value.
 
     enum Color {
         RED, GREEN, BLUE
@@ -1152,7 +1155,8 @@ For the moment, only simple enums are implemented. Enum is a list of constants,
     assertEquals( Color.valueOf("GREEN"), Color.GREEN )
     >>> void
 
-Enums are serialized as ordinals. Please note that due to caching, serialized string arrays could be even more compact than enum arrays, until `Lynon.encodeTyped` will be implemented.
+Enums are serialized as ordinals. Please note that due to caching, serialized string arrays could be even more compact
+than enum arrays, until `Lynon.encodeTyped` will be implemented.
 
 # Comments
 
@@ -1266,6 +1270,7 @@ Typical set of String functions includes:
 |--------------------|------------------------------------------------------------|
 | lower()            | change case to unicode upper                               |
 | upper()            | change case to unicode lower                               |
+| trim()             | trim space chars from both ends                            |
 | startsWith(prefix) | true if starts with a prefix                               |
 | endsWith(prefix)   | true if ends with a prefix                                 |
 | take(n)            | get a new string from up to n first characters             |
@@ -1293,7 +1298,7 @@ String literal could be multiline:
     "Hello
     World"
 
-In this case, it will be passed literally ot "hello\n    World". But, if there are
+In this case, it will be passed literally ot "hello\n World". But, if there are
 several lines with common left indent, it will be removed, also, forst and last lines,
 if blank, will be removed too, for example:
 
@@ -1305,7 +1310,8 @@ if blank, will be removed too, for example:
     >>> This is a second line.
     >>> void
 
-- as expected, empty lines and common indent were removed. It is much like kotlin's `""" ... """.trimIndent()` technique, but simpler ;)
+- as expected, empty lines and common indent were removed. It is much like kotlin's `""" ... """.trimIndent()`
+  technique, but simpler ;)
 
 # Built-in functions
 
@@ -1326,7 +1332,6 @@ See [math functions](math.md). Other general purpose functions are:
 | cached(builder)                              | remembers builder() on first invocation and return it then |
 | let, also, apply, run                        | see above, flow controls                                   |
 
-
 # Built-in constants
 
 | name                                | description                  |

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Parser.kt

@@ -433,10 +433,10 @@ private class Parser(fromPos: Pos) {
                 '\\' -> {
                     pos.advance() ?: raise("unterminated string")
                     when (currentChar) {
-                        'n' -> sb.append('\n')
+                        'n' -> {sb.append('\n'); pos.advance()}
-                        'r' -> sb.append('\r')
+                        'r' -> {sb.append('\r'); pos.advance()}
-                        't' -> sb.append('\t')
+                        't' -> {sb.append('\t'); pos.advance()}
-                        '"' -> sb.append('"')
+                        '"' -> {sb.append('"'); pos.advance()}
                         else -> sb.append('\\').append(currentChar)
                     }
                 }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/ScriptError.kt

@@ -33,6 +33,6 @@ open class ScriptError(val pos: Pos, val errorMessage: String, cause: Throwable?
 
 class ScriptFlowIsNoMoreCollected: Exception()
 
-class ExecutionError(val errorObject: ObjException) : ScriptError(errorObject.scope.pos, errorObject.message)
+class ExecutionError(val errorObject: ObjException) : ScriptError(errorObject.scope.pos, errorObject.message.value)
 
 class ImportException(pos: Pos, message: String) : ScriptError(pos, message)

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/Obj.kt

@@ -85,7 +85,7 @@ open class Obj {
         scope: Scope,
         name: String,
         args: Arguments = Arguments.EMPTY,
-        onNotFoundResult: (()->Obj?)? = null
+        onNotFoundResult: (() -> Obj?)? = null
     ): Obj {
         return objClass.getInstanceMemberOrNull(name)?.value?.invoke(
             scope,
@@ -116,11 +116,28 @@ open class Obj {
         return invokeInstanceMethod(scope, "contains", other).toBool()
     }
 
-    suspend open fun toString(scope: Scope): ObjString {
+    /**
+     * Default toString implementation:
+     *
+     * - if the object is a string, returns it
+     * - otherwise, if not [calledFromLyng], calls Lyng override `toString()` if exists
+     * - otherwise, meaning either called from Lyng `toString`, or there is no
+     *   Lyng override, returns the object's Kotlin variant of `toString()
+     *
+     * Note on kotlin's `toString()`: it is preferred to use this, 'scoped` version,
+     * as it can execute Lyng code using the scope and being suspending one.
+     *
+     * @param scope the scope where the string representation was requested
+     * @param calledFromLyng true if called from Lyng's `toString`. Normally this parameter should be ignored,
+     *      but it is used to avoid endless recursion in [Obj.toString] base implementation
+     */
+    suspend open fun toString(scope: Scope,calledFromLyng: Boolean=false): ObjString {
         return if (this is ObjString) this
-        else invokeInstanceMethod(scope, "toString") {
+        else if( !calledFromLyng ) {
-            ObjString(this.toString())
+            invokeInstanceMethod(scope, "toString") {
-        } as ObjString
+                ObjString(this.toString())
+            } as ObjString
+        } else { ObjString(this.toString()) }
     }
 
     /**
@@ -292,7 +309,7 @@ open class Obj {
 
         val rootObjectType = ObjClass("Obj").apply {
             addFn("toString", true) {
-                ObjString(thisObj.toString())
+                thisObj.toString(this, true)
             }
             addFn("inspect", true) {
                 thisObj.inspect(this).toObj()
@@ -485,19 +502,26 @@ data class ObjNamespace(val name: String) : Obj() {
     }
 }
 
+/**
+ * note on [getStackTrace]. If [useStackTrace] is not null, it is used instead. Otherwise, it is calculated
+ * from the current scope which is treated as exception scope. It is used to restore serialized
+ * exception with stack trace; the scope of the de-serialized exception is not valid
+ * for stack unwinding.
+ */
 open class ObjException(
     val exceptionClass: ExceptionClass,
     val scope: Scope,
-    val message: String,
+    val message: ObjString,
-    @Suppress("unused") val extraData: Obj = ObjNull
+    @Suppress("unused") val extraData: Obj = ObjNull,
+    val useStackTrace: ObjList? = null
 ) : Obj() {
     constructor(name: String, scope: Scope, message: String) : this(
         getOrCreateExceptionClass(name),
         scope,
-        message
+        ObjString(message)
     )
 
-    private val cachedStackTrace = CachedExpression<ObjList>()
+    private val cachedStackTrace = CachedExpression(initialValue = useStackTrace)
 
     suspend fun getStackTrace(): ObjList {
         return cachedStackTrace.get {
@@ -505,9 +529,9 @@ open class ObjException(
             val cls = scope.get("StackTraceEntry")!!.value as ObjClass
             var s: Scope? = scope
             var lastPos: Pos? = null
-            while( s != null ) {
+            while (s != null) {
                 val pos = s.pos
-                if( pos != lastPos && !pos.currentLine.isEmpty() ) {
+                if (pos != lastPos && !pos.currentLine.isEmpty()) {
                     result.list += cls.callWithArgs(
                         scope,
                         pos.source.objSourceName,
@@ -523,7 +547,7 @@ open class ObjException(
         }
     }
 
-    constructor(scope: Scope, message: String) : this(Root, scope, message)
+    constructor(scope: Scope, message: String) : this(Root, scope, ObjString(message))
 
     fun raise(): Nothing {
         throw ExecutionError(this)
@@ -531,13 +555,17 @@ open class ObjException(
 
     override val objClass: ObjClass = exceptionClass
 
-    override fun toString(): String {
+    override suspend fun toString(scope: Scope,calledFromLyng: Boolean): ObjString {
-        return "ObjException:${objClass.className}:${scope.pos}@${hashCode().encodeToHex()}"
+        val at = getStackTrace().list.firstOrNull()?.toString(scope)
+            ?: ObjString("(unknown)")
+        return ObjString("${objClass.className}: $message at $at")
     }
 
     override suspend fun serialize(scope: Scope, encoder: LynonEncoder, lynonType: LynonType?) {
-        encoder.encodeAny(scope, ObjString(exceptionClass.name))
+        encoder.encodeAny(scope, exceptionClass.classNameObj)
-        encoder.encodeAny(scope, ObjString(message))
+        encoder.encodeAny(scope, message)
+        encoder.encodeAny(scope, extraData)
+        encoder.encodeAny(scope, getStackTrace())
     }
 
 
@@ -545,18 +573,40 @@ open class ObjException(
 
         class ExceptionClass(val name: String, vararg parents: ObjClass) : ObjClass(name, *parents) {
             override suspend fun callOn(scope: Scope): Obj {
-                val message = scope.args.getOrNull(0)?.toString() ?: name
+                val message = scope.args.getOrNull(0)?.toString(scope) ?: ObjString(name)
                 return ObjException(this, scope, message)
             }
 
             override fun toString(): String = "ExceptionClass[$name]@${hashCode().encodeToHex()}"
+
+            override suspend fun deserialize(scope: Scope, decoder: LynonDecoder, lynonType: LynonType?): Obj {
+                return try {
+                    val lyngClass = decoder.decodeAnyAs<ObjString>(scope).value.let {
+                        ((scope[it] ?: scope.raiseIllegalArgument("Unknown exception class: $it"))
+                            .value as? ExceptionClass)
+                            ?: scope.raiseIllegalArgument("Not an exception class: $it")
+                    }
+                    ObjException(
+                        lyngClass,
+                        scope,
+                        decoder.decodeAnyAs<ObjString>(scope),
+                        decoder.decodeAny(scope),
+                        decoder.decodeAnyAs<ObjList>(scope)
+                    )
+                } catch (e: ScriptError) {
+                    throw e
+                } catch (e: Exception) {
+                    e.printStackTrace()
+                    scope.raiseIllegalArgument("Failed to deserialize exception: ${e.message}")
+                }
+            }
         }
 
         val Root = ExceptionClass("Throwable").apply {
             addConst("message", statement {
                 (thisObj as ObjException).message.toObj()
             })
-            addFn("getStackTrace") {
+            addFn("stackTrace") {
                 (thisObj as ObjException).getStackTrace()
             }
         }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjRange.kt

@@ -26,7 +26,7 @@ class ObjRange(val start: Obj?, val end: Obj?, val isEndInclusive: Boolean) : Ob
 
     override val objClass: ObjClass = type
 
-    override suspend fun toString(scope: Scope): ObjString {
+    override suspend fun toString(scope: Scope,calledFromLyng: Boolean): ObjString {
         val result = StringBuilder()
         result.append("${start?.inspect(scope) ?: '∞'} ..")
         if (!isEndInclusive) result.append('<')

lynglib/src/commonMain/kotlin/net/sergeych/lyng/stdlib_included/root_lyng.kt

@@ -118,6 +118,13 @@ class StackTraceEntry(
         "%s:%d:%d: %s"(sourceName, line, column, sourceString.trim())
     }
 }
+
+fun Exception.printStackTrace() {
+    println(this)
+    for( entry in stackTrace() ) {
+        println("\tat "+entry)
+    }
+}
     
 """.trimIndent()
 

lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonDecoder.kt

@@ -63,6 +63,19 @@ open class LynonDecoder(val bin: BitInput, val settings: LynonSettings = LynonSe
         }
     }
 
+    /**
+     * Decode any object with [decodeAny] and cast it to [T] or raise Lyng's class cast error
+     * with [Scope.raiseClassCastError].
+     *
+     * @return T typed Lyng object
+     */
+    suspend inline fun <reified T : Obj> decodeAnyAs(scope: Scope): T {
+        val x = decodeAny(scope)
+        return (x as? T) ?: scope.raiseClassCastError(
+                "Expected ${T::class.simpleName} but got $x"
+            )
+    }
+
     private suspend fun decodeClassObj(scope: Scope): ObjClass {
         val className = decodeObject(scope, ObjString.type, null) as ObjString
         return scope.get(className.value)?.value?.let {
@@ -72,7 +85,7 @@ open class LynonDecoder(val bin: BitInput, val settings: LynonSettings = LynonSe
         } ?: scope.raiseSymbolNotFound("can't deserialize: not found type $className")
     }
 
-    suspend fun decodeAnyList(scope: Scope,fixedSize: Int?=null): MutableList<Obj> {
+    suspend fun decodeAnyList(scope: Scope, fixedSize: Int? = null): MutableList<Obj> {
         return if (bin.getBit() == 1) {
             // homogenous
             val type = LynonType.entries[getBitsAsInt(4)]

lynglib/src/commonMain/lyng/stdlib/Iterable.lyng

@@ -1,16 +0,0 @@
-
-fun Iterable.filter( predicate ) {
-    flow {
-        for( item in this )
-            if( predicate(item) )
-                emit(item)
-    }
-}
-
-fun Iterable.drop(n) {
-    require( n >= 0, "drop amount must be non-negative")
-    var count = 0
-    filter {
-        count++ < N
-    }
-}

lynglib/src/commonTest/kotlin/ScriptTest.kt

@@ -2035,7 +2035,7 @@ class ScriptTest {
 
     @Test
     fun testThrowFromKotlin() = runTest {
-        val c = Scope()
+        val c = Script.newScope()
         c.addFn("callThrow") {
             raiseIllegalArgument("fromKotlin")
         }
@@ -3112,13 +3112,18 @@ class ScriptTest {
                     require(false)
                 }
                 catch (e) {
-                    println(e)
+                    println(e.stackTrace)
-                    println(e.getStackTrace())
+                    e.printStackTrace()
-                    for( t in e.getStackTrace() ) {
+                    val coded = Lynon.encode(e)
-                        println(t)
+                    val decoded = Lynon.decode(coded)
-                    }
+                    assertEquals( e::class, decoded::class )
-//                    val coded = Lynon.encode(e)
+                    assertEquals( e.stackTrace, decoded.stackTrace )
-//                    println(coded.toDump())
+                    assertEquals( e.message, decoded.message )
+                    println("-------------------- e")
+                    println(e.toString())
+                    println("-------------------- dee")
+                    println(decoded.toString())
+//                    assertEquals( e.toString(), decoded.toString() )
                 }
                 """.trimIndent()
         )