refs #35 serialize any: null, int, real, boolean, Instant. Added unary minus as general operator, not only to numbers. Instant truncation (nice for serialization)

2025-07-17 23:18:00 +03:00 · 2025-07-17 23:18:00 +03:00 · 6ab438b1f6
commit 6ab438b1f6
parent cffe4eaffc
14 changed files with 208 additions and 51 deletions
--- a/docs/time.md
+++ b/docs/time.md
@ -3,17 +3,22 @@
 Lyng date and time support requires importing `lyng.time` packages. Lyng uses simple yet modern time object models:

 - `Instant` class for time stamps with platform-dependent resolution
- `Duration` to represent amount of time not depending on the calendar, e.g. in absolute units (milliseconds, seconds, hours, days)
+- `Duration` to represent amount of time not depending on the calendar, e.g. in absolute units (milliseconds, seconds,
+  hours, days)

 ## Time instant: `Instant`

-Represent some moment of time not depending on the calendar (calendar for example may b e changed, daylight saving time can be for example introduced or dropped). It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin. Some moment of time; not the calendar date.
+Represent some moment of time not depending on the calendar (calendar for example may b e changed, daylight saving time
+can be for example introduced or dropped). It is similar to `TIMESTAMP` in SQL or `Instant` in Kotlin. Some moment of
+time; not the calendar date.

-Instant is comparable to other Instant. Subtracting instants produce `Duration`, period in time that is not dependent on the calendar, e.g. absolute time period.
+Instant is comparable to other Instant. Subtracting instants produce `Duration`, period in time that is not dependent on
+the calendar, e.g. absolute time period.

 It is possible to add or subtract `Duration` to and from `Instant`, that gives another `Instant`.

-Instants are converted to and from `Real` number of seconds before or after Unix Epoch, 01.01.1970. Constructor with single number parameter constructs from such number of seconds,
+Instants are converted to and from `Real` number of seconds before or after Unix Epoch, 01.01.1970. Constructor with
+single number parameter constructs from such number of seconds,
 and any instance provide `.epochSeconds` member:

    import lyng.time
@ -61,13 +66,13 @@ The resolution of system clock could be more precise and double precision real n
 ## Getting the max precision

 Normally, subtracting instants gives precision to microseconds, which is well inside the jitter
-the language VM adds. Still `Instant()` captures most precise system timer at hand and provide
-inner value of 12 bytes, up to nanoseconds (hopefully). To access it use:
+the language VM adds. Still `Instant()` or `Instant.now()` capture most precise system timer at hand and provide inner
+value of 12 bytes, up to nanoseconds (hopefully). To access it use:

    import lyng.time

    // capture time
-    val now = Instant()
+    val now = Instant.now()

    // this is Int value, number of whole epoch 
    // milliseconds to the moment, it fits 8 bytes Int well
@ -84,6 +89,22 @@ inner value of 12 bytes, up to nanoseconds (hopefully). To access it use:
    assertEquals( now.epochSeconds, nanos * 1e-9 + seconds )
    >>> void

+## Truncating to more realistic precision
+
+Full precision Instant is way too long and impractical to store, especially when serializing,
+so it is possible to truncate it to milliseconds, microseconds or seconds:
+
+    import lyng.time
+    import lyng.serialization
+
+    // max supported size (now microseconds for serialized value):
+    assert( Lynon.encode(Instant.now()).size in [8,9] )
+    // shorter: milliseconds only
+    assertEquals( 7, Lynon.encode(Instant.now().truncateToMillisecond()).size )
+    // truncated to seconds, good for file mtime, etc:
+    assertEquals( 6, Lynon.encode(Instant.now().truncateToSecond()).size )
+    >>> void
+
 ## Formatting instants

 You can freely use `Instant` in string formatting. It supports usual sprintf-style formats:
@ -104,27 +125,36 @@ You can freely use `Instant` in string formatting. It supports usual sprintf-sty
        assertEquals( unixEpoch, "Now is %d since unix epoch"(now.epochSeconds.toInt()) )
        >>> void

-See the [complete list of available formats](https://github.com/sergeych/mp_stools?tab=readme-ov-file#datetime-formatting) and the [formatting reference](https://github.com/sergeych/mp_stools?tab=readme-ov-file#printf--sprintf): it all works in Lyng as `"format"(args...)`!
+See
+the [complete list of available formats](https://github.com/sergeych/mp_stools?tab=readme-ov-file#datetime-formatting)
+and the [formatting reference](https://github.com/sergeych/mp_stools?tab=readme-ov-file#printf--sprintf): it all works
+in Lyng as `"format"(args...)`!

 ## Instant members

-| member                   | description                                             |
-|--------------------------|---------------------------------------------------------|
-| epochSeconds: Real       | positive or negative offset in seconds since Unix epoch |
-| epochWholeSeconds: Int   | same, but in _whole seconds_. Slightly faster           |
-| nanosecondsOfSecond: Int | offset from epochWholeSeconds in nanos (1)              |
-| isDistantFuture: Bool    | true if it `Instant.distantFuture`                      |
-| isDistantPast: Bool      | true if it `Instant.distantPast`                        |
+| member                         | description                                             |
+|--------------------------------|---------------------------------------------------------|
+| epochSeconds: Real             | positive or negative offset in seconds since Unix epoch |
+| epochWholeSeconds: Int         | same, but in _whole seconds_. Slightly faster           |
+| nanosecondsOfSecond: Int       | offset from epochWholeSeconds in nanos (1)              |
+| isDistantFuture: Bool          | true if it `Instant.distantFuture`                      |
+| isDistantPast: Bool            | true if it `Instant.distantPast`                        |
+| truncateToSecond: Intant       | create new instnce truncated to second                  |  
+| truncateToMillisecond: Instant | truncate new instance with to millisecond               |
+| truncateToMicrosecond: Instant | truncate new instance to microsecond                    |

 (1)
-: The value of nanoseconds is to be added to `epochWholeSeconds` to get exact time point. It is in 0..999_999_999 range. The precise time instant value therefore needs as for now 12 bytes integer; we might use bigint later (it is planned to be added)
+: The value of nanoseconds is to be added to `epochWholeSeconds` to get exact time point. It is in 0..999_999_999 range.
+The precise time instant value therefore needs as for now 12 bytes integer; we might use bigint later (it is planned to
+be added)

 ## Class members

-| member                         | description                                             |
-|--------------------------------|---------------------------------------------------------|
-| Instant.distantPast: Instant   | most distant instant in past                            |
-| Instant.distantFuture: Instant | most distant instant in future                          |
+| member                         | description                                  |
+|--------------------------------|----------------------------------------------|
+| Instant.now()                  | create new instance with current system time |
+| Instant.distantPast: Instant   | most distant instant in past                 |
+| Instant.distantFuture: Instant | most distant instant in future               |

 # `Duraion` class

@ -153,7 +183,8 @@ Duration can be converted from numbers, like `5.minutes` and so on. Extensions a

 The bigger time units like months or years are calendar-dependent and can't be used with `Duration`.

-Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration` instance:
+Each duration instance can be converted to number of any of these time units, as `Real` number, if `d` is a `Duration`
+instance:

 - `d.microseconds`
 - `d.milliseconds`
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/Compiler.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/Compiler.kt
@ -715,7 +715,7 @@ class Compiler(
        }
    }

-    private fun parseAccessor(): Accessor? {
+    private suspend fun parseAccessor(): Accessor? {
        // could be: literal
        val t = cc.next()
        return when (t.type) {
@ -737,8 +737,14 @@ class Compiler(
            }

            Token.Type.MINUS -> {
-                val n = parseNumber(false)
-                Accessor { n.asReadonly }
+                parseNumberOrNull(false)?.let { n ->
+                    Accessor { n.asReadonly }
+                } ?: run {
+                    val n = parseTerm() ?: throw ScriptError(t.pos, "Expecting expression after unary minus")
+                    Accessor {
+                        n.getter.invoke(it).value.negate(it).asReadonly
+                    }
+                }
            }

            Token.Type.ID -> {
@ -769,7 +775,8 @@ class Compiler(
        }
    }

-    private fun parseNumber(isPlus: Boolean): Obj {
+    private fun parseNumberOrNull(isPlus: Boolean): Obj? {
+        val pos = cc.savePos()
        val t = cc.next()
        return when (t.type) {
            Token.Type.INT, Token.Type.HEX -> {
@ -783,11 +790,16 @@ class Compiler(
            }

            else -> {
-                throw ScriptError(t.pos, "expected number")
+                cc.restorePos(pos)
+                null
            }
        }
    }

+    private fun parseNumber(isPlus: Boolean): Obj {
+        return parseNumberOrNull(isPlus) ?: throw ScriptError(cc.currentPos(), "Expecting number")
+    }
+
    /**
     * Parse keyword-starting statement.
     * @return parsed statement or null if, for example. [id] is not among keywords
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/Obj.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/Obj.kt
@ -138,6 +138,10 @@ open class Obj {
        scope.raiseNotImplemented()
    }

+    open suspend fun negate(scope: Scope): Obj {
+        scope.raiseNotImplemented()
+    }
+
    open suspend fun mul(scope: Scope, other: Obj): Obj {
        scope.raiseNotImplemented()
    }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInstant.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInstant.kt
@ -5,8 +5,9 @@ import kotlinx.datetime.Instant
 import kotlinx.datetime.isDistantFuture
 import kotlinx.datetime.isDistantPast
 import net.sergeych.lyng.Scope
+import net.sergeych.lynon.LynonSettings

-class ObjInstant(val instant: Instant) : Obj() {
+class ObjInstant(val instant: Instant,val truncateMode: LynonSettings.InstantTruncateMode=LynonSettings.InstantTruncateMode.Microsecond) : Obj() {
    override val objClass: ObjClass get() = type

    override fun toString(): String {
@ -102,10 +103,31 @@ class ObjInstant(val instant: Instant) : Obj() {
            addFn("nanosecondsOfSecond") {
                ObjInt(thisAs<ObjInstant>().instant.nanosecondsOfSecond.toLong())
            }
+            addFn("truncateToSecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(Instant.fromEpochSeconds(t.epochSeconds), LynonSettings.InstantTruncateMode.Second)
+            }
+            addFn("truncateToMillisecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(
+                    Instant.fromEpochSeconds(t.epochSeconds, t.nanosecondsOfSecond / 1_000_000 * 1_000_000),
+                    LynonSettings.InstantTruncateMode.Millisecond
+                )
+            }
+            addFn("truncateToMicrosecond") {
+                val t = thisAs<ObjInstant>().instant
+                ObjInstant(
+                    Instant.fromEpochSeconds(t.epochSeconds, t.nanosecondsOfSecond / 1_000 * 1_000),
+                    LynonSettings.InstantTruncateMode.Microsecond
+                )
+            }
            // class members

            addClassConst("distantFuture", distantFuture)
            addClassConst("distantPast", distantPast)
+            addClassFn("now") {
+                ObjInstant(Clock.System.now())
+            }
 //            addFn("epochMilliseconds") {
 //                ObjInt(instant.toEpochMilliseconds())
 //            }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInt.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjInt.kt
@ -97,6 +97,10 @@ class ObjInt(var value: Long,override val isConst: Boolean = false) : Obj(), Num
        return value == other.value
    }

+    override suspend fun negate(scope: Scope): Obj {
+        return ObjInt(-value)
+    }
+
    override suspend fun serialize(scope: Scope, encoder: LynonEncoder) {
        encoder.encodeSigned(value)
    }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjReal.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjReal.kt
@ -61,6 +61,10 @@ data class ObjReal(val value: Double) : Obj(), Numeric {
        return value == other.value
    }

+    override suspend fun negate(scope: Scope): Obj {
+        return ObjReal(-value)
+    }
+
    override suspend fun serialize(scope: Scope, encoder: LynonEncoder) {
        encoder.encodeReal(value)
    }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjString.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lyng/obj/ObjString.kt
@ -88,8 +88,8 @@ data class ObjString(val value: String) : Obj() {
        val type = object : ObjClass("String") {
            override fun deserialize(scope: Scope, decoder: LynonDecoder): Obj =
                ObjString(
-                    decoder.unpackBinaryData()?.decodeToString()
-                        ?: scope.raiseError("unexpected end of data")
+                    decoder.unpackBinaryData().decodeToString()
+//                        ?: scope.raiseError("unexpected end of data")
                )
        }.apply {
            addFn("toInt") {
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lynon/BitInput.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lynon/BitInput.kt
@ -82,5 +82,9 @@ interface BitInput {
    fun decompressStringOrNull(): String? = decompressOrNull()?.decodeToString()

    fun decompressString(): String = decompress().decodeToString()
+    fun unpackDouble(): Double {
+        val bits = getBits(64)
+        return Double.fromBits(bits.toLong())
+    }
 }

--- a/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonDecoder.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonDecoder.kt
@ -1,36 +1,57 @@
 package net.sergeych.lynon

+import kotlinx.datetime.Instant
 import net.sergeych.lyng.Scope
-import net.sergeych.lyng.obj.Obj
-import net.sergeych.lyng.obj.ObjClass
-import net.sergeych.lyng.obj.ObjInt
-import net.sergeych.lyng.obj.ObjNull
+import net.sergeych.lyng.obj.*

-open class LynonDecoder(val bin: BitInput,val settings: LynonSettings = LynonSettings.default) {
+open class LynonDecoder(val bin: BitInput, val settings: LynonSettings = LynonSettings.default) {

    val cache = mutableListOf<Obj>()

    inline fun decodeCached(f: LynonDecoder.() -> Obj): Obj {
-        return if( bin.getBit() == 0 ) {
+        return if (bin.getBit() == 0) {
            // unpack and cache
            f().also {
-                if( settings.shouldCache(it) ) cache.add(it)
+                if (settings.shouldCache(it)) cache.add(it)
            }
-        }
-        else {
+        } else {
            // get cache reference
            val size = sizeInBits(cache.size)
-            val id = bin.getBitsOrNull(size)?.toInt() ?: throw RuntimeException("Invalid object id: unexpected end of stream")
-            if( id >= cache.size ) throw RuntimeException("Invalid object id: $id should be in 0..<${cache.size}")
+            val id = bin.getBitsOrNull(size)?.toInt()
+                ?: throw RuntimeException("Invalid object id: unexpected end of stream")
+            if (id >= cache.size) throw RuntimeException("Invalid object id: $id should be in 0..<${cache.size}")
            cache[id]
        }
    }

    fun decodeAny(scope: Scope): Obj = decodeCached {
        val type = LynonType.entries[bin.getBits(4).toInt()]
-        return when(type) {
+        return when (type) {
            LynonType.Null -> ObjNull
            LynonType.Int0 -> ObjInt.Zero
+            LynonType.IntPositive -> ObjInt(bin.unpackUnsigned().toLong())
+            LynonType.IntNegative -> ObjInt(-bin.unpackUnsigned().toLong())
+            LynonType.Bool -> ObjBool(bin.getBit() == 1)
+            LynonType.Real -> ObjReal(bin.unpackDouble())
+            LynonType.Instant -> {
+                val mode = LynonSettings.InstantTruncateMode.entries[bin.getBits(2).toInt()]
+                when (mode) {
+                    LynonSettings.InstantTruncateMode.Microsecond -> ObjInstant(
+                        Instant.fromEpochSeconds(
+                            bin.unpackSigned(), bin.unpackUnsigned().toInt() * 1000
+                        )
+                    )
+                    LynonSettings.InstantTruncateMode.Millisecond -> ObjInstant(
+                        Instant.fromEpochMilliseconds(
+                            bin.unpackSigned()
+                        )
+                    )
+                    LynonSettings.InstantTruncateMode.Second -> ObjInstant(
+                        Instant.fromEpochSeconds(bin.unpackSigned())
+                    )
+                }
+            }
+
            else -> {
                scope.raiseNotImplemented("lynon type $type")
            }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonEncoder.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonEncoder.kt
@ -1,9 +1,7 @@
 package net.sergeych.lynon

 import net.sergeych.lyng.Scope
-import net.sergeych.lyng.obj.Obj
-import net.sergeych.lyng.obj.ObjInt
-import net.sergeych.lyng.obj.ObjNull
+import net.sergeych.lyng.obj.*

 enum class LynonType {
    Null,
@ -69,6 +67,29 @@ open class LynonEncoder(val bout: BitOutput,val settings: LynonSettings = LynonS
                        }
                    }
                }
+                is ObjBool -> {
+                    putType(LynonType.Bool)
+                    encodeBoolean(value.value)
+                }
+                is ObjReal -> {
+                    putType(LynonType.Real)
+                    encodeReal(value.value)
+                }
+                is ObjInstant -> {
+                    putType(LynonType.Instant)
+                    bout.putBits(value.truncateMode.ordinal, 2)
+                    // todo: favor truncation mode from ObjInstant
+                    when(value.truncateMode) {
+                        LynonSettings.InstantTruncateMode.Millisecond ->
+                            encodeSigned(value.instant.toEpochMilliseconds())
+                        LynonSettings.InstantTruncateMode.Second ->
+                            encodeSigned(value.instant.epochSeconds)
+                        LynonSettings.InstantTruncateMode.Microsecond -> {
+                            encodeSigned(value.instant.epochSeconds)
+                            encodeUnsigned(value.instant.nanosecondsOfSecond.toULong() / 1000UL)
+                        }
+                    }
+                }
                else -> {
                    TODO()
                }
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonSettings.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lynon/LynonSettings.kt
@ -6,7 +6,12 @@ import net.sergeych.lyng.obj.ObjInt
 import net.sergeych.lyng.obj.ObjNull
 import kotlin.math.absoluteValue

-open class LynonSettings() {
+open class LynonSettings {
+    enum class InstantTruncateMode {
+        Second,
+        Millisecond,
+        Microsecond
+    }

    open fun shouldCache(obj: Any): Boolean = when (obj) {
        is ObjChar -> false
--- a/lynglib/src/commonMain/kotlin/net/sergeych/lynon/MemoryBitOutput.kt
+++ b/lynglib/src/commonMain/kotlin/net/sergeych/lynon/MemoryBitOutput.kt
@ -52,8 +52,10 @@ class BitArray(val bytes: UByteArray, val lastByteBits: Int) : BitList {
        return result.toString()
    }

+    @Suppress("unused")
    fun asByteArray(): ByteArray = bytes.asByteArray()

+    @Suppress("unused")
    fun asUbyteArray(): UByteArray = bytes

    companion object {
@ -82,10 +84,10 @@ class BitArray(val bytes: UByteArray, val lastByteBits: Int) : BitList {
 * added by [putBit] will be stored in the bit 0x01 of the first byte, the second bit
 * in the bit 0x02 of the first byte, etc.
 *
- * This allow automatic fill of the last byte with zeros. This is important when
+ * This allows automatic fill of the last byte with zeros. This is important when
 * using bytes stored from [asByteArray] or [asUbyteArray]. When converting to
- * bytes, automatic padding to byte size is applied. With such bit order, constrinting
- * [BitInput] to read from [asByteArray] result only provides 0 to 7 extra zeroes bits
+ * bytes, automatic padding to byte size is applied. With such bit order, constructing
+ * [BitInput] to read from [ByteArray.toUByteArray] result only provides 0 to 7 extra zeroes bits
 * at teh end which is often acceptable. To avoid this, use [toBitArray]; the [BitArray]
 * stores exact number of bits and [BitArray.toBitInput] provides [BitInput] that
 * decodes exactly same bits.
--- a/lynglib/src/commonTest/kotlin/ScriptTest.kt
+++ b/lynglib/src/commonTest/kotlin/ScriptTest.kt
@ -2649,4 +2649,12 @@ class ScriptTest {
        """.trimIndent())
    }

+    @Test
+    fun testRangeToList() = runTest {
+        val x = eval("""(1..10).toList()""") as ObjList
+        assertEquals(listOf(1,2,3,4,5,6,7,8,9,10), x.list.map { it.toInt() })
+        val y = eval("""(-2..3).toList()""") as ObjList
+        println(y.list)
+    }
+
 }
--- a/lynglib/src/jvmTest/kotlin/LynonTests.kt
+++ b/lynglib/src/jvmTest/kotlin/LynonTests.kt
@ -303,10 +303,29 @@ class LynonTests {


    @Test
-    fun testIntsNulls() = runTest{
+    fun testUnaryMinus() = runTest{
        eval("""
-            import lyng.serialization
-            assertEquals( null, Lynon.decode(Lynon.encode(null)) )
+            assertEquals( -1 * π, 0 - π )
+            assertEquals( -1 * π, -π )
+            """.trimIndent())
+    }
+    
+    @Test
+    fun testIntsNulls() = runTest{
+        testScope().eval("""
+            testEncode(null)
+            testEncode(0)
+            testEncode(47)
+            testEncode(-21)
+            testEncode(true)
+            testEncode(false)
+            testEncode(1.22345)
+            testEncode(-π)
+            
+            import lyng.time
+            testEncode(Instant.now().truncateToSecond())
+            testEncode(Instant.now().truncateToMillisecond())
+            testEncode(Instant.now().truncateToMicrosecond())
        """.trimIndent())
    }