refs #35 serialize any: null, int, real, boolean, Instant. Added unary minus as general operator, not only to numbers. Instant truncation (nice for serialization)

docs/time.md

@@ -3,17 +3,22 @@
 Lyng date and time support requires importing `lyng.time` packages. Lyng uses simple yet modern time object models:
 
 - `Instant` class for time stamps with platform-dependent resolution
-- `Duration` to represent amount of time not depending on the calendar, e.g. in absolute units (milliseconds, seconds, hours, days)
+- `Duration` to represent amount of time not depending on the calendar, e.g. in absolute units (milliseconds, seconds,
+  hours, days)
 
 ## Time instant: `Instant`
 
-Represent some moment of time not depending on the calendar (calendar for example may b e changed, daylight saving time can be for example introduced or dropped). It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin. Some moment of time; not the calendar date.
+Represent some moment of time not depending on the calendar (calendar for example may b e changed, daylight saving time
+can be for example introduced or dropped). It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin. Some moment of
+time; not the calendar date.
 
-Instant is comparable to other Instant. Subtracting instants produce `Duration`, period in time that is not dependent on the calendar, e.g. absolute time period.
+Instant is comparable to other Instant. Subtracting instants produce `Duration`, period in time that is not dependent on
+the calendar, e.g. absolute time period.
 
 It is possible to add or subtract `Duration` to and from `Instant`, that gives another `Instant`.
 
-Instants are converted to and from `Real` number of seconds before or after Unix Epoch, 01.01.1970. Constructor with single number parameter constructs from such number of seconds,
+Instants are converted to and from `Real` number of seconds before or after Unix Epoch, 01.01.1970. Constructor with
+single number parameter constructs from such number of seconds,
 and any instance provide `.epochSeconds` member:
 
     import lyng.time
@@ -61,13 +66,13 @@ The resolution of system clock could be more precise and double precision real n
 ## Getting the max precision
 
 Normally, subtracting instants gives precision to microseconds, which is well inside the jitter
-the language VM adds. Still `Instant()` captures most precise system timer at hand and provide
+the language VM adds. Still `Instant()` or `Instant.now()` capture most precise system timer at hand and provide inner
-inner value of 12 bytes, up to nanoseconds (hopefully). To access it use:
+value of 12 bytes, up to nanoseconds (hopefully). To access it use:
 
     import lyng.time
 
     // capture time
-    val now = Instant()
+    val now = Instant.now()
 
     // this is Int value, number of whole epoch 
     // milliseconds to the moment, it fits 8 bytes Int well
@@ -84,6 +89,22 @@ inner value of 12 bytes, up to nanoseconds (hopefully). To access it use:
     assertEquals( now.epochSeconds, nanos * 1e-9 + seconds )
     >>> void
 
+## Truncating to more realistic precision
+
+Full precision Instant is way too long and impractical to store, especially when serializing,
+so it is possible to truncate it to milliseconds, microseconds or seconds:
+
+    import lyng.time
+    import lyng.serialization
+
+    // max supported size (now microseconds for serialized value):
+    assert( Lynon.encode(Instant.now()).size in [8,9] )
+    // shorter: milliseconds only
+    assertEquals( 7, Lynon.encode(Instant.now().truncateToMillisecond()).size )
+    // truncated to seconds, good for file mtime, etc:
+    assertEquals( 6, Lynon.encode(Instant.now().truncateToSecond()).size )
+    >>> void
+
 ## Formatting instants
 
 You can freely use `Instant` in string formatting. It supports usual sprintf-style formats:
@@ -104,27 +125,36 @@ You can freely use `Instant` in string formatting. It supports usual sprintf-sty
         assertEquals( unixEpoch, "Now is %d since unix epoch"(now.epochSeconds.toInt()) )
         >>> void
 
-See the [complete list of available formats](https://github.com/sergeych/mp_stools?tab=readme-ov-file#datetime-formatting) and the [formatting reference](https://github.com/sergeych/mp_stools?tab=readme-ov-file#printf--sprintf): it all works in Lyng as `"format"(args...)`!
+See
+the [complete list of available formats](https://github.com/sergeych/mp_stools?tab=readme-ov-file#datetime-formatting)
+and the [formatting reference](https://github.com/sergeych/mp_stools?tab=readme-ov-file#printf--sprintf): it all works
+in Lyng as `"format"(args...)`!
 
 ## Instant members
 
-| member                   | description                                             |
+| member                         | description                                             |
-|--------------------------|---------------------------------------------------------|
+|--------------------------------|---------------------------------------------------------|
-| epochSeconds: Real       | positive or negative offset in seconds since Unix epoch |
+| epochSeconds: Real             | positive or negative offset in seconds since Unix epoch |
-| epochWholeSeconds: Int   | same, but in _whole seconds_. Slightly faster           |
+| epochWholeSeconds: Int         | same, but in _whole seconds_. Slightly faster           |
-| nanosecondsOfSecond: Int | offset from epochWholeSeconds in nanos (1)              |
+| nanosecondsOfSecond: Int       | offset from epochWholeSeconds in nanos (1)              |
-| isDistantFuture: Bool    | true if it `Instant.distantFuture`                      |
+| isDistantFuture: Bool          | true if it `Instant.distantFuture`                      |
-| isDistantPast: Bool      | true if it `Instant.distantPast`                        |
+| isDistantPast: Bool            | true if it `Instant.distantPast`                        |
+| truncateToSecond: Intant       | create new instnce truncated to second                  |  
+| truncateToMillisecond: Instant | truncate new instance with to millisecond               |
+| truncateToMicrosecond: Instant | truncate new instance to microsecond                    |
 
 (1)
-: The value of nanoseconds is to be added to `epochWholeSeconds` to get exact time point. It is in 0..999_999_999 range. The precise time instant value therefore needs as for now 12 bytes integer; we might use bigint later (it is planned to be added)
+: The value of nanoseconds is to be added to `epochWholeSeconds` to get exact time point. It is in 0..999_999_999 range.
+The precise time instant value therefore needs as for now 12 bytes integer; we might use bigint later (it is planned to
+be added)
 
 ## Class members
 
-| member                         | description                                             |
+| member                         | description                                  |
-|--------------------------------|---------------------------------------------------------|
+|--------------------------------|----------------------------------------------|
-| Instant.distantPast: Instant   | most distant instant in past                            |
+| Instant.now()                  | create new instance with current system time |
-| Instant.distantFuture: Instant | most distant instant in future                          |
+| Instant.distantPast: Instant   | most distant instant in past                 |
+| Instant.distantFuture: Instant | most distant instant in future               |
 
 # `Duraion` class
 
@@ -153,7 +183,8 @@ Duration can be converted from numbers, like `5.minutes` and so on. Extensions a
 
 The bigger time units like months or years are calendar-dependent and can't be used with `Duration`.
 
-Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration` instance:
+Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration`
+instance:
 
 - `d.microseconds`
 - `d.milliseconds`

lynglib/src/commonMain/kotlin/net/sergeych/lyng/Compiler.kt

@@ -715,7 +715,7 @@ class Compiler(
         }
     }
 
-    private fun parseAccessor(): Accessor? {
+    private suspend fun parseAccessor(): Accessor? {
         // could be: literal
         val t = cc.next()
         return when (t.type) {
@@ -737,8 +737,14 @@ class Compiler(
             }
 
             Token.Type.MINUS -> {
-                val n = parseNumber(false)
+                parseNumberOrNull(false)?.let { n ->
-                Accessor { n.asReadonly }
+                    Accessor { n.asReadonly }
+                } ?: run {
+                    val n = parseTerm() ?: throw ScriptError(t.pos, "Expecting expression after unary minus")
+                    Accessor {
+                        n.getter.invoke(it).value.negate(it).asReadonly
+                    }
+                }
             }
 
             Token.Type.ID -> {
@@ -769,7 +775,8 @@ class Compiler(
         }
     }
 
-    private fun parseNumber(isPlus: Boolean): Obj {
+    private fun parseNumberOrNull(isPlus: Boolean): Obj? {
+        val pos = cc.savePos()
         val t = cc.next()
         return when (t.type) {
             Token.Type.INT, Token.Type.HEX -> {
@@ -783,11 +790,16 @@ class Compiler(
             }
 
             else -> {
-                throw ScriptError(t.pos, "expected number")
+                cc.restorePos(pos)
+                null
             }
         }
     }
 
+    private fun parseNumber(isPlus: Boolean): Obj {
+        return parseNumberOrNull(isPlus) ?: throw ScriptError(cc.currentPos(), "Expecting number")
+    }
+
     /**
      * Parse keyword-starting statement.
      * @return parsed statement or null if, for example. [id] is not among keywords

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/Obj.kt

@@ -138,6 +138,10 @@ open class Obj {
         scope.raiseNotImplemented()
     }
 
+    open suspend fun negate(scope: Scope): Obj {
+        scope.raiseNotImplemented()
+    }
+
     open suspend fun mul(scope: Scope, other: Obj): Obj {
         scope.raiseNotImplemented()
     }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInstant.kt

@@ -5,8 +5,9 @@ import kotlinx.datetime.Instant
 import kotlinx.datetime.isDistantFuture
 import kotlinx.datetime.isDistantPast
 import net.sergeych.lyng.Scope
+import net.sergeych.lynon.LynonSettings
 
-class ObjInstant(val instant: Instant) : Obj() {
+class ObjInstant(val instant: Instant,val truncateMode: LynonSettings.InstantTruncateMode=LynonSettings.InstantTruncateMode.Microsecond) : Obj() {
     override val objClass: ObjClass get() = type
 
     override fun toString(): String {
@@ -102,10 +103,31 @@ class ObjInstant(val instant: Instant) : Obj() {
             addFn("nanosecondsOfSecond") {
                 ObjInt(thisAs<ObjInstant>().instant.nanosecondsOfSecond.toLong())
             }
+            addFn("truncateToSecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(Instant.fromEpochSeconds(t.epochSeconds), LynonSettings.InstantTruncateMode.Second)
+            }
+            addFn("truncateToMillisecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(
+                    Instant.fromEpochSeconds(t.epochSeconds, t.nanosecondsOfSecond / 1_000_000 * 1_000_000),
+                    LynonSettings.InstantTruncateMode.Millisecond
+                )
+            }
+            addFn("truncateToMicrosecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(
+                    Instant.fromEpochSeconds(t.epochSeconds, t.nanosecondsOfSecond / 1_000 * 1_000),
+                    LynonSettings.InstantTruncateMode.Microsecond
+                )
+            }
             // class members
 
             addClassConst("distantFuture", distantFuture)
             addClassConst("distantPast", distantPast)
+            addClassFn("now") {
+                ObjInstant(Clock.System.now())
+            }
 //            addFn("epochMilliseconds") {
 //                ObjInt(instant.toEpochMilliseconds())
 //            }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInt.kt

@@ -97,6 +97,10 @@ class ObjInt(var value: Long,override val isConst: Boolean = false) : Obj(), Num
         return value == other.value
     }
 
+    override suspend fun negate(scope: Scope): Obj {
+        return ObjInt(-value)
+    }
+
     override suspend fun serialize(scope: Scope, encoder: LynonEncoder) {
         encoder.encodeSigned(value)
     }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjReal.kt

@@ -61,6 +61,10 @@ data class ObjReal(val value: Double) : Obj(), Numeric {
         return value == other.value
     }
 
+    override suspend fun negate(scope: Scope): Obj {
+        return ObjReal(-value)
+    }
+
     override suspend fun serialize(scope: Scope, encoder: LynonEncoder) {
         encoder.encodeReal(value)
     }

lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjString.kt

@@ -88,8 +88,8 @@ data class ObjString(val value: String) : Obj() {
         val type = object : ObjClass("String") {
             override fun deserialize(scope: Scope, decoder: LynonDecoder): Obj =
                 ObjString(
-                    decoder.unpackBinaryData()?.decodeToString()
+                    decoder.unpackBinaryData().decodeToString()
-                        ?: scope.raiseError("unexpected end of data")
+//                        ?: scope.raiseError("unexpected end of data")
                 )
         }.apply {
             addFn("toInt") {

lynglib/src/commonMain/kotlin/net/sergeych/lynon/BitInput.kt

@@ -82,5 +82,9 @@ interface BitInput {
     fun decompressStringOrNull(): String? = decompressOrNull()?.decodeToString()
 
     fun decompressString(): String = decompress().decodeToString()
+    fun unpackDouble(): Double {
+        val bits = getBits(64)
+        return Double.fromBits(bits.toLong())
+    }
 }
 

lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonDecoder.kt

@@ -1,36 +1,57 @@
 package net.sergeych.lynon
 
+import kotlinx.datetime.Instant
 import net.sergeych.lyng.Scope
-import net.sergeych.lyng.obj.Obj
+import net.sergeych.lyng.obj.*
-import net.sergeych.lyng.obj.ObjClass
-import net.sergeych.lyng.obj.ObjInt
-import net.sergeych.lyng.obj.ObjNull
 
-open class LynonDecoder(val bin: BitInput,val settings: LynonSettings = LynonSettings.default) {
+open class LynonDecoder(val bin: BitInput, val settings: LynonSettings = LynonSettings.default) {
 
     val cache = mutableListOf<Obj>()
 
     inline fun decodeCached(f: LynonDecoder.() -> Obj): Obj {
-        return if( bin.getBit() == 0 ) {
+        return if (bin.getBit() == 0) {
             // unpack and cache
             f().also {
-                if( settings.shouldCache(it) ) cache.add(it)
+                if (settings.shouldCache(it)) cache.add(it)
             }
-        }
+        } else {
-        else {
             // get cache reference
             val size = sizeInBits(cache.size)
-            val id = bin.getBitsOrNull(size)?.toInt() ?: throw RuntimeException("Invalid object id: unexpected end of stream")
+            val id = bin.getBitsOrNull(size)?.toInt()
-            if( id >= cache.size ) throw RuntimeException("Invalid object id: $id should be in 0..<${cache.size}")
+                ?: throw RuntimeException("Invalid object id: unexpected end of stream")
+            if (id >= cache.size) throw RuntimeException("Invalid object id: $id should be in 0..<${cache.size}")
             cache[id]
         }
     }
 
     fun decodeAny(scope: Scope): Obj = decodeCached {
         val type = LynonType.entries[bin.getBits(4).toInt()]
-        return when(type) {
+        return when (type) {
             LynonType.Null -> ObjNull
             LynonType.Int0 -> ObjInt.Zero
+            LynonType.IntPositive -> ObjInt(bin.unpackUnsigned().toLong())
+            LynonType.IntNegative -> ObjInt(-bin.unpackUnsigned().toLong())
+            LynonType.Bool -> ObjBool(bin.getBit() == 1)
+            LynonType.Real -> ObjReal(bin.unpackDouble())
+            LynonType.Instant -> {
+                val mode = LynonSettings.InstantTruncateMode.entries[bin.getBits(2).toInt()]
+                when (mode) {
+                    LynonSettings.InstantTruncateMode.Microsecond -> ObjInstant(
+                        Instant.fromEpochSeconds(
+                            bin.unpackSigned(), bin.unpackUnsigned().toInt() * 1000
+                        )
+                    )
+                    LynonSettings.InstantTruncateMode.Millisecond -> ObjInstant(
+                        Instant.fromEpochMilliseconds(
+                            bin.unpackSigned()
+                        )
+                    )
+                    LynonSettings.InstantTruncateMode.Second -> ObjInstant(
+                        Instant.fromEpochSeconds(bin.unpackSigned())
+                    )
+                }
+            }
+
             else -> {
                 scope.raiseNotImplemented("lynon type $type")
             }

lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonEncoder.kt

@@ -1,9 +1,7 @@
 package net.sergeych.lynon
 
 import net.sergeych.lyng.Scope
-import net.sergeych.lyng.obj.Obj
+import net.sergeych.lyng.obj.*
-import net.sergeych.lyng.obj.ObjInt
-import net.sergeych.lyng.obj.ObjNull
 
 enum class LynonType {
     Null,
@@ -69,6 +67,29 @@ open class LynonEncoder(val bout: BitOutput,val settings: LynonSettings = LynonS
                         }
                     }
                 }
+                is ObjBool -> {
+                    putType(LynonType.Bool)
+                    encodeBoolean(value.value)
+                }
+                is ObjReal -> {
+                    putType(LynonType.Real)
+                    encodeReal(value.value)
+                }
+                is ObjInstant -> {
+                    putType(LynonType.Instant)
+                    bout.putBits(value.truncateMode.ordinal, 2)
+                    // todo: favor truncation mode from ObjInstant
+                    when(value.truncateMode) {
+                        LynonSettings.InstantTruncateMode.Millisecond ->
+                            encodeSigned(value.instant.toEpochMilliseconds())
+                        LynonSettings.InstantTruncateMode.Second ->
+                            encodeSigned(value.instant.epochSeconds)
+                        LynonSettings.InstantTruncateMode.Microsecond -> {
+                            encodeSigned(value.instant.epochSeconds)
+                            encodeUnsigned(value.instant.nanosecondsOfSecond.toULong() / 1000UL)
+                        }
+                    }
+                }
                 else -> {
                     TODO()
                 }

lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonSettings.kt

@@ -6,7 +6,12 @@ import net.sergeych.lyng.obj.ObjInt
 import net.sergeych.lyng.obj.ObjNull
 import kotlin.math.absoluteValue
 
-open class LynonSettings() {
+open class LynonSettings {
+    enum class InstantTruncateMode {
+        Second,
+        Millisecond,
+        Microsecond
+    }
 
     open fun shouldCache(obj: Any): Boolean = when (obj) {
         is ObjChar -> false

lynglib/src/commonMain/kotlin/net/sergeych/lynon/MemoryBitOutput.kt

@@ -52,8 +52,10 @@ class BitArray(val bytes: UByteArray, val lastByteBits: Int) : BitList {
         return result.toString()
     }
 
+    @Suppress("unused")
     fun asByteArray(): ByteArray = bytes.asByteArray()
 
+    @Suppress("unused")
     fun asUbyteArray(): UByteArray = bytes
 
     companion object {
@@ -82,10 +84,10 @@ class BitArray(val bytes: UByteArray, val lastByteBits: Int) : BitList {
  * added by [putBit] will be stored in the bit 0x01 of the first byte, the second bit
  * in the bit 0x02 of the first byte, etc.
  *
- * This allow automatic fill of the last byte with zeros. This is important when
+ * This allows automatic fill of the last byte with zeros. This is important when
  * using bytes stored from [asByteArray] or [asUbyteArray]. When converting to
- * bytes, automatic padding to byte size is applied. With such bit order, constrinting
+ * bytes, automatic padding to byte size is applied. With such bit order, constructing
- * [BitInput] to read from [asByteArray] result only provides 0 to 7 extra zeroes bits
+ * [BitInput] to read from [ByteArray.toUByteArray] result only provides 0 to 7 extra zeroes bits
  * at teh end which is often acceptable. To avoid this, use [toBitArray]; the [BitArray]
  * stores exact number of bits and [BitArray.toBitInput] provides [BitInput] that
  * decodes exactly same bits.

lynglib/src/commonTest/kotlin/ScriptTest.kt

@@ -2649,4 +2649,12 @@ class ScriptTest {
         """.trimIndent())
     }
 
+    @Test
+    fun testRangeToList() = runTest {
+        val x = eval("""(1..10).toList()""") as ObjList
+        assertEquals(listOf(1,2,3,4,5,6,7,8,9,10), x.list.map { it.toInt() })
+        val y = eval("""(-2..3).toList()""") as ObjList
+        println(y.list)
+    }
+
 }

lynglib/src/jvmTest/kotlin/LynonTests.kt

@@ -303,10 +303,29 @@ class LynonTests {
 
 
     @Test
-    fun testIntsNulls() = runTest{
+    fun testUnaryMinus() = runTest{
         eval("""
-            import lyng.serialization
+            assertEquals( -1 * π, 0 - π )
-            assertEquals( null, Lynon.decode(Lynon.encode(null)) )
+            assertEquals( -1 * π, -π )
+            """.trimIndent())
+    }
+    
+    @Test
+    fun testIntsNulls() = runTest{
+        testScope().eval("""
+            testEncode(null)
+            testEncode(0)
+            testEncode(47)
+            testEncode(-21)
+            testEncode(true)
+            testEncode(false)
+            testEncode(1.22345)
+            testEncode(-π)
+            
+            import lyng.time
+            testEncode(Instant.now().truncateToSecond())
+            testEncode(Instant.now().truncateToMillisecond())
+            testEncode(Instant.now().truncateToMicrosecond())
         """.trimIndent())
     }