# Proposal: Ktor-Backed Networking Modules For `lyngio` Status: Draft Date: 2026-04-02 Owner: `lyngio` ## Context `lyngio` currently provides optional modules for filesystem, process execution, and console access. Networking is still missing. The first networking release should optimize for: - broad multiplatform reach from day one - a stable Lyng-native API surface - small implementation cost by building on existing, well-tested KMP libraries - explicit capability discovery, because not every transport exists on every platform The proposed implementation base is: - `io.ktor:ktor-client-*` for HTTP/HTTPS - Ktor WebSockets client on top of the HTTP client for `ws` and `wss` - `io.ktor:ktor-network` for TCP client/server sockets and UDP datagrams - no `ktor-server` The critical design constraint is that Lyng must not expose Ktor types directly. Ktor is an implementation dependency, not the public scripting model. ## Goals - Add first-release networking support with a practical breadth of features: - HTTP/HTTPS client - WebSocket client (`ws` / `wss`) - TCP client socket - TCP server socket where supported - UDP socket with a minimal datagram API where supported - Keep the Lyng-facing API uniform across platforms. - Provide capability discovery functions so scripts can branch safely. - Keep declarations in `.lyng` files as the source of truth. - Preserve room for later lower-level or non-universal socket APIs. ## Non-goals (phase 1) - exposing Ktor classes, channels, frames, engines, or plugins directly to Lyng - HTTP server support - TLS configuration beyond default secure client behavior - fully general stream/channel APIs for every transport - advanced socket tuning beyond a minimal useful set - promising identical runtime support on every platform ## Proposed module split Use three Lyng modules instead of one monolith: - `lyng.io.http` - `lyng.io.ws` - `lyng.io.net` Rationale: - HTTP and WebSocket are much more portable than raw sockets. - TCP/UDP belong in a transport module, not in the HTTP client API. - Each module can have its own capability checks while still sharing common address and security types underneath. - The user-facing import structure stays clear and unsurprising. ## Uniform capability model Every module should expose explicit capability discovery. Recommended shape: - `Http.isSupported(): Bool` - `Ws.isSupported(): Bool` - `Net.isSupported(): Bool` - `Net.isTcpAvailable(): Bool` - `Net.isTcpServerAvailable(): Bool` - `Net.isUdpAvailable(): Bool` Optional richer form for later: - `Http.details(): HttpSupport` - `Ws.details(): WsSupport` - `Net.details(): NetSupport` For v1, booleans are enough. Scripts should be able to write: ```lyng if (Net.isTcpServerAvailable()) { val server = Net.tcpListen(4040) } ``` This is preferable to hard-failing during import or construction. ## Design principles ### 1. Lyng-native API, Ktor-backed implementation Public Lyng objects should be small and script-oriented. Do not expose: - `HttpClient` - `HttpResponse` - `ByteReadChannel` - `ByteWriteChannel` - `Frame` - Ktor engine names - Ktor pipeline/plugin concepts Instead, expose a narrow stable surface implemented internally via Ktor. ### 2. Minimal but broad v1 The first release should support more transport kinds, but only with the smallest coherent API for each kind. That means: - HTTP: request/response, headers, text/body bytes, status, method - WebSocket: connect, send text/binary, receive text/binary, close - TCP: connect, accept, read, write, close - UDP: bind, send datagram, receive datagram, close Notably absent from v1: - interceptors - cookies/session stores beyond defaults - multipart builders - HTTP/2 tuning knobs - ping/pong tuning for WebSockets - socket option explosion ### 3. Uniform shapes where practical Across transports, prefer the same ideas: - `isOpen()` - `close()` - `localAddress()` - `remoteAddress()` when meaningful - `read(...)`, `write(...)` - `receive()`, `send(...)` for datagrams/messages - `isSupported()` / capability methods on module singletons This keeps discovery predictable without forcing false sameness where semantics differ. ## Proposed Lyng modules ### `lyng.io.http` Purpose: - high-level HTTP/HTTPS client backed by Ktor client Recommended declaration sketch: ```lyng package lyng.io.http extern class HttpHeaders : Map { fun get(name: String): String? fun getAll(name: String): List fun names(): List } extern class HttpRequest { var method: String var url: String var headers: Map var bodyText: String? var bodyBytes: Buffer? var timeoutMillis: Int? } extern class HttpResponse { val status: Int val statusText: String val headers: HttpHeaders fun text(): String fun bytes(): Buffer } extern object Http { fun isSupported(): Bool fun request(req: HttpRequest): HttpResponse fun get(url: String, headers...): HttpResponse fun post(url: String, bodyText: String = "", contentType: String? = null, headers...): HttpResponse fun postBytes(url: String, body: Buffer, contentType: String? = null, headers...): HttpResponse } ``` Notes: - `HttpRequest` is mutable because it is ergonomic in scripts. - `headers` is intentionally simple in v1: `Map` for programmatic request construction. - `HttpHeaders` is primarily for response headers, where repeated names matter. - `text()` and `bytes()` should cache decoded content inside the response object. - For convenience methods, `headers...` should accept `MapEntry` values such as `"X-Token" => "abc"` and `[key, value]` pairs, then normalize them internally into request headers. ### `lyng.io.ws` Purpose: - WebSocket client built on Ktor client WebSockets Recommended declaration sketch: ```lyng package lyng.io.ws extern class WsMessage { val isText: Bool val text: String? val data: Buffer? } extern class WsSession { fun isOpen(): Bool fun url(): String fun sendText(text: String): void fun sendBytes(data: Buffer): void fun receive(): WsMessage? fun close(code: Int = 1000, reason: String = ""): void } extern object Ws { fun isSupported(): Bool fun connect(url: String, headers...): WsSession } ``` Notes: - Keep frames hidden. Lyng sees messages. - `receive()` returns `null` on clean close. - v1 supports text and binary messages only. - `headers...` should follow the same rules as `Http.get(...)`: `MapEntry` values or `[key, value]` pairs. ### `lyng.io.net` Purpose: - transport sockets backed by `ktor-network` Recommended declaration sketch: ```lyng package lyng.io.net enum IpVersion { IPV4, IPV6 } extern class SocketAddress { val host: String val port: Int val ipVersion: IpVersion val resolved: Bool fun toString(): String } extern class Datagram { val data: Buffer val address: SocketAddress } extern class TcpSocket { fun isOpen(): Bool fun localAddress(): SocketAddress fun remoteAddress(): SocketAddress fun read(maxBytes: Int = 65536): Buffer? fun readLine(): String? fun write(data: Buffer): void fun writeUtf8(text: String): void fun flush(): void fun close(): void } extern class TcpServer { fun isOpen(): Bool fun localAddress(): SocketAddress fun accept(): TcpSocket fun close(): void } extern class UdpSocket { fun isOpen(): Bool fun localAddress(): SocketAddress fun receive(maxBytes: Int = 65536): Datagram? fun send(data: Buffer, host: String, port: Int): void fun close(): void } extern object Net { fun isSupported(): Bool fun isTcpAvailable(): Bool fun isTcpServerAvailable(): Bool fun isUdpAvailable(): Bool fun resolve(host: String, port: Int): List fun tcpConnect( host: String, port: Int, timeoutMillis: Int? = null, noDelay: Bool = true ): TcpSocket fun tcpListen( port: Int, host: String? = null, backlog: Int = 128, reuseAddress: Bool = true ): TcpServer fun udpBind( port: Int = 0, host: String? = null, reuseAddress: Bool = true ): UdpSocket } ``` Notes: - UDP is included in v1, but only with a minimal datagram API. - `udpBind` should bind an ephemeral port by default. - TCP server methods exist only where the backend supports them, but the declaration stays uniform. ## Example usage HTTP: ```lyng import lyng.io.http if (!Http.isSupported()) throw IllegalStateException("http is not supported here") val r = Http.get( "https://example.com", "Accept" => "text/plain", "X-Trace" => "demo" ) println(r.status) println(r.text()) ``` WebSocket: ```lyng import lyng.io.ws if (Ws.isSupported()) { val ws = Ws.connect( "wss://echo.example/ws", "Authorization" => "Bearer " ) try { ws.sendText("hello") val msg = ws.receive() if (msg != null && msg.isText) println(msg.text) } finally { ws.close() } } ``` TCP: ```lyng import lyng.io.net if (Net.isTcpAvailable()) { val s = Net.tcpConnect("example.com", 80) try { s.writeUtf8("GET / HTTP/1.1\r\nHost: example.com\r\nConnection: close\r\n\r\n") s.flush() while (true) { val line = s.readLine() if (line == null) break println(line) } } finally { s.close() } } ``` TCP server: ```lyng import lyng.io.net if (Net.isTcpServerAvailable()) { val server = Net.tcpListen(4040, host="127.0.0.1") println("listening on " + server.localAddress()) try { while (server.isOpen()) { val client = server.accept() launch { try { println("accepted " + client.remoteAddress()) val requestLine = client.readLine() if (requestLine != null) println("request: " + requestLine) while (true) { val line = client.readLine() if (line == null || line == "") break } client.writeUtf8("hello from lyng server\n") client.flush() } finally { client.close() } } } } finally { server.close() } } ``` UDP: ```lyng import lyng.io.net if (Net.isUdpAvailable()) { val udp = Net.udpBind() try { udp.send(Buffer("ping"), "127.0.0.1", 4041) val d = udp.receive() if (d != null) println("from " + d.address + ": " + d.data) } finally { udp.close() } } ``` ## Platform support model The interface is uniform. Support is platform-dependent. Initial intent: | Platform | HTTP | WS client | TCP client | TCP server | UDP | | :--- | :---: | :---: | :---: | :---: | :---: | | JVM | yes | yes | yes | yes | yes | | Android | yes | yes | likely yes | likely yes | likely yes | | Native Linux/macOS | yes | yes | target | target | target | | Native Windows | yes | yes | target | target | target | | NodeJS | yes | engine-dependent | target | target | target | | Browser / Wasm | yes | engine-dependent | no or target-specific | no | no | Interpretation rules: - import succeeds when the module is installed - construction/operations may still fail if the specific feature is unavailable - discovery methods must let scripts avoid that path cleanly - `isSupported()` means the module has at least one useful implementation on the current runtime ## Security model Use separate policies per module to avoid muddy authorization. Recommended policies: - `HttpAccessPolicy` - `WsAccessPolicy` - `NetAccessPolicy` Suggested operations: ```kotlin sealed interface HttpAccessOp { data class Request(val method: String, val url: String) : HttpAccessOp } sealed interface WsAccessOp { data class Connect(val url: String) : WsAccessOp data class Send(val url: String, val bytes: Int, val isText: Boolean) : WsAccessOp data class Receive(val url: String) : WsAccessOp } sealed interface NetAccessOp { data class Resolve(val host: String, val port: Int) : NetAccessOp data class TcpConnect(val host: String, val port: Int) : NetAccessOp data class TcpListen(val host: String?, val port: Int, val backlog: Int) : NetAccessOp data class TcpAccept(val localHost: String?, val localPort: Int, val remoteHost: String, val remotePort: Int) : NetAccessOp data class TcpRead(val remoteHost: String, val remotePort: Int, val requestedBytes: Int) : NetAccessOp data class TcpWrite(val remoteHost: String, val remotePort: Int, val bytes: Int) : NetAccessOp data class UdpBind(val host: String?, val port: Int) : NetAccessOp data class UdpSend(val host: String, val port: Int, val bytes: Int) : NetAccessOp data class UdpReceive(val localHost: String?, val localPort: Int, val requestedBytes: Int) : NetAccessOp } ``` This is intentionally strict. It lets embedders enforce: - allow HTTP but forbid raw sockets - allow outbound TCP but forbid listening sockets - allow UDP only on loopback - allow WebSockets only to whitelisted hosts ## Error mapping Like other `lyngio` modules, host exceptions should be mapped into Lyng runtime errors. Recommended v1 mapping: - policy denial -> illegal operation - malformed URL / invalid host / invalid port -> illegal argument - connect failure / bind failure / timeout / protocol failure -> illegal state Later, if needed, add typed Lyng exceptions such as: - `HttpException` - `WebSocketException` - `NetworkException` - `TimeoutException` ## Implementation outline Recommended layout: - `lyngio/stdlib/lyng/io/http.lyng` - `lyngio/stdlib/lyng/io/ws.lyng` - `lyngio/stdlib/lyng/io/net.lyng` - `lyngio/src/commonMain/kotlin/net/sergeych/lyng/io/http/LyngHttpModule.kt` - `lyngio/src/commonMain/kotlin/net/sergeych/lyng/io/ws/LyngWsModule.kt` - `lyngio/src/commonMain/kotlin/net/sergeych/lyng/io/net/LyngNetModule.kt` - `lyngio/src/commonMain/kotlin/net/sergeych/lyngio/http/...` - `lyngio/src/commonMain/kotlin/net/sergeych/lyngio/ws/...` - `lyngio/src/commonMain/kotlin/net/sergeych/lyngio/net/...` Module installers should mirror existing `lyngio` patterns: ```kotlin fun createHttpModule(policy: HttpAccessPolicy, scope: Scope): Boolean fun createWsModule(policy: WsAccessPolicy, scope: Scope): Boolean fun createNetModule(policy: NetAccessPolicy, scope: Scope): Boolean ``` As with `lyng.io.console`, the registrar should evaluate embedded `.lyng` declarations into module scope. ## Why this is better than the earlier raw-sockets-first plan - it delivers useful networking sooner - it covers HTTP and WebSocket immediately, not as future work - it reuses mature KMP networking code instead of creating a transport stack from scratch - it still leaves room for a later low-level socket API if Ktor abstractions prove insufficient ## Open questions 1. Should `Http` use static convenience methods only in v1, or also expose a reusable `HttpClient`-like Lyng object later? 2. Should `Ws.connect(...)` accept only URL plus headers, or also subprotocols in v1? 3. Should `TcpSocket.readExact(...)` be included from the start, or wait until there is a real use case? 4. Should `UdpSocket.send(...)` accept `SocketAddress` overloads in v1, or only `host` and `port`? 5. Should `isSupported()` return true when only a subset of the module works, or should module support be stricter? ## Recommended implementation order 1. `lyng.io.http` 2. `lyng.io.ws` 3. `lyng.io.net` TCP client 4. `lyng.io.net` TCP server 5. `lyng.io.net` UDP This order matches both practical usefulness and implementation risk.