6.2 KiB
Numeric transport priority — protocol 2
import net.sergeych.kiloparsec.call
import net.sergeych.kiloparsec.push
remote.call(command, arguments, priority = 100)
remote.push(notification, arguments, priority = 0)
Larger signed Int values go first within the allowed nonce range. Equal
priorities use nonce order for encrypted packets and FIFO for unnumbered packets. Existing
call/push entry points remain unchanged and use zero unless invoked inside an
explicit priority call context. The overloads are extension functions, preserving
existing RemoteInterface implementers. A coroutine context carries the selected
value through the encrypted RPC wrappers; it becomes serialized outer-block data.
The transport owns a nonce-aware outgoing writer and submits one block at a time to the adapter, preserving the selected order through the carrier. Waiting for a response does not occupy the writer: calls retain independent request IDs and response waiters. Both transport responses and errors copy the request's priority, including encrypted RPC errors returned inside an outer response. Pushes have no normal response.
Wire compatibility
The legacy block tag and body are unchanged. Nonzero priority is appended as four big-endian two's-complement bytes. Zero omits the trailer entirely. The new reader defaults to zero only when the trailer is absent; a partial trailer is malformed.
This relies on each Transport.Block having its own finite byte-array frame. TCP's existing length prefix, WebSocket messages and UDP data blocks provide that boundary. The old decoder consumes the legacy body and ignores trailing bytes. Do not use this EOF-terminated extension to concatenate unframed Block objects or embed them directly in a positional collection; wrap each block in its own bytes. The priority trailer itself retains its legacy encoding. The secure handshake requires protocol 2; this does not enable communication with protocol-1 peers. BiPack @Extendable is deliberately not added: it would insert a new field-count prefix before existing fields and break old readers.
Adapter behavior
TCP, WebSocket client/server, and UDP use a per-connection priority writer.
The writer reads the legacy header lengths and skips payloads to locate priority;
it does not deserialize/copy/decrypt the payload or guess from the last four bytes.
Transport.Device.send(block) defaults to the existing output channel for custom
or legacy devices. Built-in ProxyDevice-based adapters override it to enter their
writer directly. The old output channel remains available, bridged into that writer;
new transports should call send so pending senders are visible to priority ordering.
Transport callers now receive nonce-aware scheduling even with a custom FIFO
adapter. Direct adapter users retain the adapter-level scheduling behavior.
Queue admission is bounded to 128 waiting blocks and 32 MiB (plus the block currently being written). Overflow fails explicitly rather than allocating an unbounded backlog. Send awaits its adapter write callback, not a remote RPC reply. Queued cancellation removes the block; cancellation cannot recall a write already started. Closing the writer releases pending senders. There is no rate limiter or batch-collection delay. WebSocket writes flush after each block to avoid excessive look-ahead in the engine's FIFO. Socket/TLS buffers and network loss can still delay control behind bytes already submitted; priority is not a preemption guarantee.
Nonce-aware admission and fairness
OutQueue.push(message, nonce, priority) records scheduling metadata locally; no
new wire fields are added. pop() finds the smallest queued nonce and selects the
highest-priority message no more than 2730 positions ahead of it. This is two
thirds of the 4096-position receive replay window. Gaps count as nonce distance,
not as transmitted packets. The bound prevents an old encrypted message from
being starved until it becomes stale. It leaves headroom for external reordering,
but does not promise acceptance after arbitrary network delay or reordering.
A transport's preparation mutex covers nonce allocation, encryption, final block encoding and queue admission. Encryption remains in the secure-layer callbacks. All encrypted paths (identity/finished, calls, pushes and replies) use this path. The mutex is released before waiting for transmission. Full queues reject admission immediately; there is no capacity wait while holding this mutex. Cancellation may leave a nonce gap but never reuses a nonce.
The transport writer serializes carrier submissions, so a downstream adapter cannot reorder these submissions again. Public direct writes to an adapter's output channel bypass this transport contract.
Large-message preparation is not preemptible by priority. A local JVM probe with concurrent 1 MiB and 1 KiB payloads measured preparation including mutex waiting at medians of about 87 ms for both sizes (maximum about 118 ms). This includes encoding and allocation, not just cipher time; it is a local observation, not a throughput or latency guarantee.
Current verification
Focused tests cover replay-window boundaries and bitmap wraparound, concurrent nonce admission in both directions, duplicate/stale delivery, local send failure recovery, cancellation/capacity/close, and an encrypted 4098-message backlog whose old low-priority message remains acceptable. The saturated encrypted-link test continues to pass. Kotlin JVM tests, JS/Wasm compilation, TypeScript tests and TypeScript-to-JVM interoperability were checked for this change.
Earlier priority-release verification
Focused JVM tests cover signed numeric ordering, stable ties, queued cancellation, bounded admission, close, absent/partial trailers, unchanged legacy bodies, and priority inheritance through encrypted concurrent calls and encrypted errors. An encrypted simulated 16 KiB/s link saturated in both directions maintains control round trips below 400 ms and aggregate utilization above 80% of full-duplex capacity. This simulation does not model kernel/TLS buffering or packet loss. JVM, Android and JS compilation and socket/resource lifecycle tests pass. Caravan also checked call/error/response fixtures with the actual packaged 0.7.1-SNAPSHOT binary and the new 0.7.2-SNAPSHOT binary in separate JVM processes.