lyng/docs/json_and_kotlin_serialization.md

Json support

Lyng now has two distinct JSON-facing layers:

• plain JSON projection:
  • Obj.toJson()
  • Obj.toJsonString()
• canonical JSON round-trip format:
  • Json.encode(value)
  • Json.decode(text)
• typed canonical JSON round-trip format:
  • Json.encodeAs(Type, value)
  • Json.decodeAs(Type, text)

Use the first when you need ordinary JSON for interop.

Use the second when you need Lyng value round-trip semantics through JSON text with no schema.

Use the third when both sides already know the Lyng type and you want the same round-trip semantics with fewer type tags in the JSON.

This distinction is intentional:

• plain JSON projection is optimized for compatibility with ordinary JSON tooling
• canonical Json.encode() is optimized for semantic fidelity to Lyng and Lynon and stays self-describing
• typed canonical Json.encodeAs() is optimized for the same fidelity when the schema is provided externally
• these goals conflict for values such as sets, exceptions, singleton objects, buffers, and maps with non-string keys

Plain JSON projection in Lyng

// in lyng
assertEquals("{\"a\":1}", {a: 1}.toJsonString())
void
>>> void

Simple classes serialization is supported:

import lyng.serialization
class Point(foo,bar) {
    val t = 42
}
// val is not serialized
assertEquals( "{\"foo\":1,\"bar\":2}", Point(1,2).toJsonString() )
>>> void

Note that mutable members are serialized by default. You can exclude any member (including constructor parameters) from JSON serialization using the @Transient attribute:

import lyng.serialization

class Point2(@Transient val foo, val bar) {
    @Transient var reason = 42
    var visible = 100
}
assertEquals( "{\"bar\":2,\"visible\":100}", Point2(1,2).toJsonString() )
>>> void

Note that if you override plain JSON serialization:

import lyng.serialization

class Point2(foo,bar) {
    var reason = 42
    // but we override json serialization:
    fun toJsonObject() {
        { "custom": true }
    }
}
// var is serialized instead
assertEquals( "{\"custom\":true}", Point2(1,2).toJsonString() )
>>> void

Custom serialization of user classes is possible by overriding toJsonObject. It must return an object which is serializable to JSON. Most often it is a map, but any object is accepted:

import lyng.serialization

class Point2(foo,bar) {
    var reason = 42
    // but we override json serialization:
    fun toJsonObject() {
        { "custom": true }
    }
}
class Custom {
    fun toJsonObject() {
        "full freedom"
    }
}
// var is serialized instead
assertEquals( "\"full freedom\"", Custom().toJsonString() )
>>> void

Please note that toJsonString should be used to get serialized string representation of the object. Don't call toJsonObject directly, it is not intended to be used outside the serialization library.

Canonical Json round-trip format

Json.encode() and Json.decode() are now the JSON equivalents of Lynon.encode() and Lynon.decode().

They still use JSON text, but they add Lyng-specific type tags where plain JSON would otherwise lose information.

When a map already fits ordinary JSON object rules, canonical JSON keeps that traditional object shape. In particular, maps with string keys are still serialized as JSON objects, not as tagged entry lists.

Example:

import lyng.serialization
import lyng.time

enum Color { Red, Green }
class Point(x,y) { var z = 42 }

val p = Point(1,2)
p.z = 99

val value = List(
    p,
    Map([1, "one"], ["two", 2]),
    Set(1,2,3),
    "hello".encodeUtf8(),
    Date(2026,4,15),
    Color.Green
)

assertEquals(value, Json.decode(Json.encode(value)))

The canonical Json format is intended for Lyng-to-Lyng transfer through JSON text.

The plain toJson() projection is intended for ordinary JSON interop.

Canonical Json.encode() should be read as the JSON analogue of Lynon.encode(): when Lynon already preserves a Lyng distinction, canonical JSON tries to preserve it too, using tags only where ordinary JSON is insufficient.

Typed canonical Json round-trip format

Json.encodeAs(Type, value) and Json.decodeAs(Type, text) use the same canonical rules, but with a declared target type available during the whole traversal.

This changes one thing only: type tags may be omitted when the declared type is already exact enough to restore the value unambiguously.

The same map rule still applies here: Map<String, T> stays a normal JSON object, while non-string-key maps fall back to canonical entry encoding.

Example:

import lyng.serialization

closed class Point(x: Int, y: Int)
closed class Segment(a: Point, b: Point)

val value = Segment(Point(0, 1), Point(2, 3))
val encoded = Json.encodeAs(Segment, value)

assertEquals("{\"a\":{\"x\":0,\"y\":1},\"b\":{\"x\":2,\"y\":3}}", encoded)
assertEquals(value, Json.decodeAs(Segment, encoded))

Subtype information is still preserved when the declared type is wider than the runtime one. For example, if a field is declared as Base but contains Derived, canonical subtype tags remain in that field.

This is why the APIs are split:

• toJson() stays plain and interop-friendly
• Json.encode() stays fully self-describing and safe to decode without a schema
• Json.encodeAs() uses the supplied schema to reduce noise, but only where that schema is sufficient

Kotlin side interfaces

The "Batteries included" principle is also applied to serialization.

• Obj.toJson() provides Kotlin JsonElement for the plain JSON projection
• Obj.toJsonString() provides plain JSON string representation
• Obj.decodeSerializableWith() and Obj.decodeSerializable() allows to decode Lyng classes as Kotlin objects using kotlinx.serialization:

/**
 * Decodes the current object into a deserialized form using the provided deserialization strategy.
 * It is based on [Obj.toJson] and uses existing Kotlin Json serialization, without string representation
 * (only `JsonElement` to carry information between Kotlin and Lyng serialization worlds), thus efficient.
 *
 * @param strategy The deserialization strategy that defines how the object should be decoded.
 * @param scope An optional scope used during deserialization to define the context. Defaults to a new instance of Scope.
 * @return The deserialized object of type T.
 */
suspend fun <T> Obj.decodeSerializableWith(strategy: DeserializationStrategy<T>, scope: Scope = Scope()): T =
    Json.decodeFromJsonElement(strategy, toJson(scope))

/**
 * Decodes a serializable object of type [T] using the provided decoding scope. The deserialization uses
 * [Obj.toJson] and existing Json based serialization ithout using actual string representation, thus
 * efficient.
 *
 * @param T The type of the object to be decoded. Must be a reified type.
 * @param scope The scope used during decoding. Defaults to a new instance of [Scope].
 */
suspend inline fun <reified T> Obj.decodeSerializable(scope: Scope = Scope()) =
    decodeSerializableWith<T>(serializer<T>(), scope)

Note that Lyng-to-Kotlin deserialization with kotlinx.serialization is based on the plain JSON projection, not the canonical Json.encode() format. It uses JsonElement as the information carrier without formatting and parsing actual JSON strings. This is why we use Json.decodeFromJsonElement instead of Json.decodeFromString.

Pitfall: JSON objects and Map<String, Any?>

Kotlin serialization does not support Map<String, Any?> as a serializable type, more general, it can't serialize Any. This in particular means that you can deserialize Kotlin Map<String, T> as long as T is @Serializable in Kotlin:

@Serializable
data class TestJson2(
    val value: Int,
    val inner: Map<String,Int>
)

@Test
fun deserializeMapWithJsonTest() = runTest {
    val session = EvalSession()
    val x = session.eval("""
        import lyng.serialization
        { value: 1, inner: { "foo": 1, "bar": 2 }}
    """.trimIndent()).decodeSerializable<TestJson2>()
    // That works perfectly well:
    assertEquals(TestJson2(1, mapOf("foo" to 1, "bar" to 2)), x)
}

But what if your map has objects of different types? The approach of using polymorphism is partially applicable, but what to do with { one: 1, two: "two" }?

The answer is simple: use JsonObject in your deserializable object. This class is capable of holding any JSON types and structures:

@Serializable
data class TestJson3(
    val value: Int,
    val inner: JsonObject
)
@Test
fun deserializeAnyMapWithJsonTest() = runTest {
    val session = EvalSession()
    val x = session.eval("""
        import lyng.serialization
        { value: 12, inner: { "foo": 1, "bar": "two" }}
    """.trimIndent()).decodeSerializable<TestJson3>()
    assertEquals(TestJson3(12, JsonObject(mapOf("foo" to JsonPrimitive(1), "bar" to Json.encodeToJsonElement("two")))), x)
}

Supported shapes

Plain JSON projection

Lyng type	JSON type	notes
Int	number
Real	number	finite values only as plain numbers
String	string
Bool	boolean
null	null
Instant	string	ISO8601 (1)
List	array	(2)
Map	object	string keys only
simple class instance	object	constructor fields + mutable vars
enum	string	entry name

Canonical Json.encode

This format can also round-trip:

• maps with non-string keys
• sets
• immutable collections
• buffers and bit buffers
• class instances
• singleton objects
• enums
• exceptions
• Date, Instant, DateTime
• non-finite reals
• void

Typed canonical Json.encodeAs

This format round-trips the same value space as canonical Json.encode, but it can emit simpler JSON for:

• closed classes and other exactly-known class fields
• enums when the enum type is known
• typed collections whose element types are known
• nested object graphs where declared field types are precise

It still falls back to canonical tagged encoding when exact runtime type information would otherwise be lost.

It does so by adding Lyng-specific type tags only when necessary.

Kotlin-side extension point for more formats

Additional formats can be exported from Kotlin modules by subclassing ObjSerializationFormatClass and registering the format in module scope with bindSerializationFormat(...).

module.bindSerializationFormat(
    object : ObjSerializationFormatClass("MyFormat") {
        override suspend fun encodeValue(scope: Scope, value: Obj): Obj = ...
        override suspend fun decodeValue(scope: Scope, encoded: Obj): Obj = ...
    }
)

This makes MyFormat.encode(...) and MyFormat.decode(...) available from Lyng after importing the module.

(1)

ISO8601 flavor 1970-05-06T06:00:00.000Z is used; number of fractional digits depends on truncation on Instant, see Instant.truncateTo... functions.

(2)

Lists may contain any values serializable by the selected JSON layer.