464 lines
51 KiB
Markdown
464 lines
51 KiB
Markdown
# Concealed Kiloparsec protocol: design record
|
|
|
|
Recorded: 2026-10-21; implementation work started 2026-10-22. Current candidate: 0.9.0-SNAPSHOT, an experimental source implementation and wire draft with [qualification evidence](CONCEALED-PROTOCOL-QUALIFICATION.md); not an audited or universally runtime-qualified protocol.
|
|
|
|
See [wire specification](CONCEALED-PROTOCOL-WIRE.md), [implementation/continuation](CONCEALED-PROTOCOL-IMPLEMENTATION.md) and the [timing assessment](CONCEALED-PROTOCOL-TIMING.md) for current code, usage, measurements and validation status.
|
|
|
|
This document preserves the conversation before further design work. It distinguishes agreed direction from assistant proposals and unresolved details. It is the continuation point for this work. The existing protocol-2 review is [REVIEW-237d404.md](REVIEW-237d404.md); its issue numbers I1–I7 concern the released implementation, not this new design.
|
|
|
|
Stable references: **D1–D14** are decisions/directions, **C1–C8** are corrections and reasoning, **Q1–Q14** are discussion questions (Q1/Q3–Q12 resolved at design level; Q2 is a specification task; Q13/Q14 are implementation/validation tasks), **V1–V7** are proposed validation steps. Do not renumber existing references; append new ones. Questions are not approval gates for routine work. Discuss them individually before freezing the wire format.
|
|
|
|
## 1. Objective and scope
|
|
|
|
### D1 — Conceal protocol metadata as well as application data
|
|
|
|
The present protocol has recognizable envelopes and exposed routing metadata. The new flavor should make its wire bytes look random, including the initial handshake, while authenticating the information needed to process records. No visible magic prefix, protocol version, command name, RPC identifier, priority, error code or sequential packet counter should be required in the concealed encoding.
|
|
|
|
This is a second connection flavor, not an instruction to replace the released protocol. Use separate configured ports/URLs for the existing and concealed flavors (resolved Q8). Endpoint configuration selects the protocol before parsing the handshake; no wire marker or autodetection is required. The original shared-endpoint autodetection idea is deferred as an optional future capability, not initial scope.
|
|
|
|
Random-looking bytes do not conceal endpoints, traffic volume, timing, packet lengths or application traffic patterns. A raw TCP connection and a WebSocket connection also expose different carrier information. No claim of universal undetectability, censorship resistance, or security against all active attacks has been established.
|
|
|
|
### D2 — KMP first, no new crypto library dependency
|
|
|
|
Scope is the Kotlin Multiplatform library. Leave the lightweight TypeScript client and its present protocol behavior unchanged. A later TS implementation is possible, but is not part of the initial implementation.
|
|
|
|
Reuse the existing libsodium bindings and Ionspin KMP bignum library. Avoid adding Monocypher or another runtime crypto library solely for Elligator. Independent implementations may be used as test references without becoming shipped dependencies.
|
|
|
|
### D3 — A pre-known, publicly distributable server encryption key
|
|
|
|
The client knows a suitable server encryption public key before the handshake. It may be obtained from a site, API, an earlier authenticated connection, or another provisioning mechanism. Confidential distribution is not required. Authentic distribution is still necessary to know which server is being authenticated.
|
|
|
|
A shared confidential entrance secret was proposed and not selected. Anyone knowing the public encryption key can construct a new encrypted HELO. This means the design does not prevent active probes by such parties merely through bootstrap encryption.
|
|
|
|
Separate encryption and signing keys are involved if we use both proofs. Their authentic association, provisioning and replacement belong to application policy (resolved Q3); automatic rotation is outside the communications protocol.
|
|
|
|
## 2. Cryptographic bootstrap
|
|
|
|
### D4 — X25519 with Elligator 2 concealment
|
|
|
|
RSA was considered, but not selected. Ordinary X25519 ephemeral key encodings are distinguishable from uniformly random 32-byte strings. Elligator 2 is the selected direction for concealing the exposed ephemeral bootstrap key.
|
|
|
|
Conceptual bootstrap layout, not yet a wire specification:
|
|
|
|
```text
|
|
32-byte Elligator representative | authentication tag | encrypted HELO
|
|
```
|
|
|
|
The discussion used a 16-byte tag and the 48-byte overhead of libsodium sealed boxes as the model. This does not mean an unchanged crypto_box_seal call implements the intended bootstrap. Exact suite, derivation encodings and authenticated data remain Q2; the response-protection choice is resolved in Q1.
|
|
|
|
The server's static encryption public key need not itself be Elligator-encoded in the packet: it is pre-known. Only ephemeral key material exposed on the wire needs concealment. Subsequent DH keys can be carried inside encryption.
|
|
|
|
Correct concealed key generation requires more than applying an inverse map to an ordinary public key:
|
|
|
|
1. Generate a fresh ephemeral secret.
|
|
2. Generate the specially randomized public point, including the required small-order component/sign handling.
|
|
3. Apply the inverse Elligator map; regenerate locally if the point is not representable.
|
|
4. Encode with the required random choices, including otherwise unused high bits.
|
|
|
|
Approximately two generation attempts are expected for a representable point. This is distinct from rejecting an invalid peer DH input: never loop regenerating our key to accommodate a malicious low-order peer key.
|
|
|
|
### D5 — Implement the mapping in Kotlin using existing arithmetic
|
|
|
|
Verified dependency graph in Kiloparsec:
|
|
|
|
```text
|
|
crypto2:0.9.0 -> com.ionspin.kotlin:bignum:0.3.10
|
|
```
|
|
|
|
BigInteger/ModularBigInteger, not BigDecimal, supply the relevant arithmetic. Existing bindings 0.9.6 expose Edwards base multiplication, point addition and X25519. Proposed split:
|
|
|
|
| Work | Component |
|
|
| --- | --- |
|
|
| Random bytes, base multiplication, point addition, X25519 | Existing libsodium |
|
|
| Edwards/Montgomery conversion, Elligator maps, encoding | Kotlin + existing bignum |
|
|
| Hashing, authentication and encryption | Existing libsodium |
|
|
|
|
Libsodium's scalar arithmetic is modulo the subgroup order; Elligator field arithmetic is modulo p = 2^255 - 19. They are not interchangeable. Its from_uniform function maps to a subgroup point; it does not provide the inverse concealed-key generation we need.
|
|
|
|
This is a feasible direction, not a tested implementation. Mixed-order point acceptance must be checked through the actual bindings. General bignum arithmetic is not assumed constant-time. Concealed intermediate points cannot automatically be treated as harmless public information. Start with a readable, independently checked prototype, then evaluate timing and production suitability. Added binary size must be measured: using already-present dependencies can retain additional previously stripped code.
|
|
|
|
### D6 — Move client identity proof into a fresh-session ACK
|
|
|
|
Agreed handshake direction:
|
|
|
|
```text
|
|
C -> S: HELO, encrypted to the known server encryption key
|
|
fresh client session-DH public key; no client identity/signature
|
|
|
|
S -> C: server hello
|
|
fresh server session-DH public key
|
|
proof relating to this HELO
|
|
server signature (agreed addition)
|
|
encrypted/authenticated with the Q1 reply key and a fresh random 24-byte IV
|
|
|
|
C -> S: ACK, encrypted/authenticated under fresh session keys
|
|
client identity public key
|
|
client signature binding this session
|
|
|
|
S -> C: acceptance, encrypted/authenticated under fresh session keys
|
|
sent after ACK verification and successful application authorization
|
|
```
|
|
|
|
Both session-DH key pairs are fresh for each connection attempt. Remove the client identity public key as well as its signature from HELO. Moving only the signature would not conceal identity against later compromise of the server's static decryption key.
|
|
|
|
The user proposed the connection token derived from the directional session-key pair in the proper order as the ACK's signed value. The current SafeKeyExchange session tag is a relevant existing mechanism, but the exact new token derivation and signature input are not frozen. Role/domain separation and handshake-context binding were agreed: cover the connection token, both session-DH public keys and selected flavor/options. Q2 remains the task of specifying exact inputs and encodings.
|
|
|
|
Once ACK arrives and verifies, the server has authenticated the client. It then completes application authorization and sends a protected acceptance response. The client reports connected only after verifying this response. This fourth handshake message is agreed (resolved Q4).
|
|
|
|
### D7 — Server proof and signature
|
|
|
|
The user proposed BLAKE2b(clientSessionDhPublicKey) as proof that the server decrypted HELO. That client public key is fresh and concealed inside HELO; it is not necessarily the exposed ephemeral bootstrap key. Under that assumption, the hash is evidence of knowing the concealed value.
|
|
|
|
The user subsequently agreed to add a server signature over the server DH key and that proof. The exact signed transcript, role label, options and identity-key association remain Q2/Q3.
|
|
|
|
Q1 is resolved: derive a domain-separated temporary reply key from the fresh client session-DH public key concealed inside HELO. Encrypt/authenticate the complete signed server response using that key and a fresh random 24-byte IV, transmitted alongside the ciphertext. This uses the concealed client DH value as bootstrap material, not the exposed Elligator representative. Retain the agreed server signature. Exact suite and derivation encodings remain Q2.
|
|
|
|
### D8 — Reject failed DH agreement at the crypto interface
|
|
|
|
Libsodium crypto_kx already rejects unacceptable low-order inputs before deriving session keys. Inspected JVM and Linux bindings propagate failures. Runtime tests through crypto2 verified rejection on both roles, including nonzero low-order encodings and high-bit aliases.
|
|
|
|
The check concerns failure of the operation / an all-zero raw DH result, not merely whether received public-key bytes contain a nonzero byte. Checking derived session keys for zero is not a substitute: those keys are hashes of the DH result and context.
|
|
|
|
The crypto2 investigation found a separate missing length guard. SafeKeyExchange now rejects peer public-key arrays whose length is not exactly 32 before entering native code. This was published as crypto2 0.9.6-SNAPSHOT to Maven Local/private Maven and committed/pushed as 70d7a13. All 67 JVM and 67 Linux Native tests passed. Its focused JS tests fail during the existing libsodium loader initialization (_sodium_init is not a function), not DH validation. Other runtimes were not qualified.
|
|
|
|
Kiloparsec has not been switched from its existing crypto2 dependency as part of this design discussion.
|
|
|
|
## 3. Established connection
|
|
|
|
### D9 — One protection layer below RPC
|
|
|
|
Do not tunnel the current encrypted L0Call protocol inside the new encrypted channel. Serialize ordinary RPC blocks, then protect them once at the channel layer.
|
|
|
|
```text
|
|
Command / RemoteInterface / LocalInterface / application scope
|
|
-> RPC calls, responses, errors, pushes and correlation
|
|
-> protected channel
|
|
-> carrier
|
|
```
|
|
|
|
Reuse application APIs and authorization hooks. A serialized Transport.Block can initially serve as the inner message. Its IDs, names, priority and error metadata are then entirely concealed and authenticated. Avoid duplicating those fields in another ProtectedMessage merely to preserve the old wrapping architecture.
|
|
|
|
Current relevant implementation points:
|
|
|
|
- Transport.kt: block serialization, correlation, pending waiters, dispatch and sending.
|
|
- KiloRemoteInterface.kt and KiloL0Interface.kt: current L0Call/protected-message wrapping.
|
|
- KiloClientConnection.kt and KiloServerConnection.kt: current handshake and lifecycle.
|
|
- KiloParams.kt: current session cipher/counter handling.
|
|
- ReplayWindow.kt, adapter/OutQueue.kt, adapter/PriorityBlockWriter.kt: replay/scheduling machinery.
|
|
|
|
The exact common interface and refactoring boundary remain Q12. Device currently carries bytes and is not, by itself, a sufficient interface for carrying local scheduling metadata.
|
|
|
|
### D10 — UDP: independent records with random 24-byte IVs
|
|
|
|
Agreed for the datagram path:
|
|
|
|
```text
|
|
random IV [24 bytes] | authentication tag [16 bytes] | ciphertext {
|
|
sequence number | RPC block | optional padding
|
|
}
|
|
```
|
|
|
|
The IV is the encryption nonce. Generate it with the CSPRNG; no session-seeded custom PRNG construction is selected. Random generation gives negligible collision probability, not strict mathematical uniqueness. For q independent 192-bit nonces under one directional key, collision probability is approximately q(q-1)/2^193.
|
|
|
|
Fresh directional session keys isolate the two directions. The concealed sequential counter performs replay tracking and queue ordering; the random IV permits independent decryption after loss/reordering. Authenticate before updating replay state, dispatching a command or consuming a pending response waiter. Duplicate/stale records are nonfatal.
|
|
|
|
Retain the preparation mutex and sequence-aware priority queue: allocate sequence, encrypt and admit atomically; await actual transmission outside the mutex. Counter exhaustion must not wrap. The existing 4096-position window and 2730 maximum local overtaking are the reuse baseline, not a guarantee against arbitrary network delay.
|
|
|
|
Q9 adds configurable random-length random-byte padding inside the protected record. No packet-loss recovery for application RPCs is implicitly added by encryption or replay tracking.
|
|
|
|
### D11 — TCP: continuous encryption plus object authentication
|
|
|
|
The user chose a continuous counter-mode/stream-cipher construction. Each direction maintains its own keystream position; arbitrary socket write/read segmentation must not affect it. Use fresh directional encryption keys and a defined initial nonce/counter; never restart the stream under reused key/nonce material.
|
|
|
|
Agreed conceptual plaintext stream:
|
|
|
|
```text
|
|
length | encoded object + optional padding | keyed BLAKE2b tag
|
|
length | encoded object + optional padding | keyed BLAKE2b tag
|
|
...
|
|
```
|
|
|
|
Encrypt every byte, including lengths and tags, with the continuous stream. No per-object IV is needed on this path. TCP's ordered delivery synchronizes the stream position. This supersedes the random-per-record-IV proposal for TCP only; UDP retains D10.
|
|
|
|
Proposed MAC input, accepted conceptually:
|
|
|
|
```text
|
|
tag = keyedBLAKE2b(macKey, sequence || exactEncodedLength || exactProtectedBodyBytes)
|
|
```
|
|
|
|
The protected body contains the encoded object and optional Q9 padding, with authenticated delimitation between them. Exact padded-body length encoding is Q2 specification work. The MAC covers the entire body, including padding and its delimiters. Use a separately derived MAC key for each direction and a 16-byte tag. The TCP object sequence may be implicit, incrementing on authenticated objects. Exact integer encodings, domain separation, cipher and limits remain Q2/Q6/Q7.
|
|
|
|
Receive pipeline:
|
|
|
|
1. Decrypt a bounded length header.
|
|
2. Reject invalid/over-limit lengths before allocation or body reading.
|
|
3. Read bounded object bytes and tag with bounded waiting.
|
|
4. Verify the MAC in constant time.
|
|
5. Only then decode BiPack and invoke RPC/application logic.
|
|
|
|
The length remains untrusted until MAC verification. A modified length can cause a bounded read, wait or connection failure; the construction does not eliminate denial of service. On invalid MAC, abort; do not scan for resynchronization or continue consuming objects.
|
|
|
|
This is a custom construction requiring validation, not an already-audited equivalent of a standard AEAD record protocol. Authenticate the exact received bytes, not a decoded-and-reserialized object. Authenticated clean closure is selected in resolved Q11.
|
|
|
|
### D12 — Priority and fairness must survive the new layering
|
|
|
|
The current queue's bounded nonce overtaking intrinsically prevents indefinite postponement of old messages. The user explicitly valued that property over a simpler lambda queue that encrypts only when popped.
|
|
|
|
TCP's continuous cipher must be advanced in actual wire order. Therefore select complete encoded objects before advancing stream encryption; serialize writes through one writer. Once bytes of an object enter the stream, another object's bytes cannot be arbitrarily interleaved with them.
|
|
|
|
Resolved Q7: reuse the existing generic OutQueue bounded-overtaking algorithm for TCP as well as UDP. For TCP, its ordering field is a monotonically increasing local admission ordinal, not the stream cipher position or implicit MAC sequence. Allocate the ordinal and enqueue atomically. Queue encoded plaintext objects; a single writer pops by priority within the bounded window, assigns the wire-order MAC sequence, authenticates/encrypts, and writes. For UDP, retain the sequence-aware ciphertext queue from D10. A large in-progress object can delay small urgent objects; cancellation after writing starts must finish that record or close the stream.
|
|
|
|
### D13 — Preserve application authorization and scope
|
|
|
|
Kotlin intentionally does not reduce server/client trust policy to a constructor list of keys. Verified remote credentials and session information must be exposed through application scope/init handlers. SecurityException and BreakConnectionException subclasses must still abort initialization.
|
|
|
|
Preserve the existing deferred readiness mechanism (resolved Q5): calls made before readiness suspend awaiting the result. RPC starts only when identities are ready and the channel is protected. Preserve existing initialization/application-policy ordering rather than introducing a new pre-establishment RPC mechanism or a blanket prohibition on RPC from init handlers. Application authorization remains enforced by its existing hooks/policy; Q4 governs the client connected notification after protected acceptance.
|
|
|
|
Protocol replay prevention is not business-operation deduplication after reconnect. A new valid request may be an intentional repeat; application idempotency remains application responsibility.
|
|
|
|
### D14 — Compatibility and release scope
|
|
|
|
Current Kiloparsec 0.8.0 was published and pushed as 009d62a / tag v0.8.0. The review's I1–I5 repairs are included; I6/I7 were closed as intentional decisions/non-defects. The concealed protocol is future work and must not be represented as part of that release.
|
|
|
|
No new wire version number, public API name or release version has been chosen. Avoid calling the new flavor "protocol 2" ambiguously: the existing released authenticated envelope already uses protocol 2.
|
|
|
|
## 4. Corrections that must not be lost
|
|
|
|
### C1 — EC encryption is hybrid encryption
|
|
|
|
X25519's 32 bytes are a public coordinate/shared-secret size, not an RSA-style plaintext block size. Public-key encryption normally exposes an ephemeral public key plus symmetric ciphertext/tag. The ephemeral bootstrap key and the client session-DH key concealed inside HELO are separate in the discussed design; their reuse/separation must be explicit in Q2.
|
|
|
|
### C2 — Ordinary public keys leave a statistical fingerprint
|
|
|
|
For normally generated X25519 encodings, canonical field range, curve membership and prime-subgroup membership give public tests. Roughly 1/32 of uniformly random 32-byte strings passes all three. This is a distinguisher against uniform bytes, not a probability that a packet is Kiloparsec. Multiple independent observations strengthen it. Poly1305 tags have no analogous public validity test under the usual assumptions.
|
|
|
|
### C3 — RSA does not automatically look uniform either
|
|
|
|
A ciphertext integer is less than the public modulus n. Random k-bit values pass that range test with probability n/2^k. RSA was not adopted as an escape from encoding distinguishers.
|
|
|
|
### C4 — Continuous XOR encryption does not authenticate
|
|
|
|
Fresh masks and fresh keys do not prevent chosen bit flips in a single ciphertext: (P XOR K XOR delta) XOR K = P XOR delta. The attack does not require keystream reuse.
|
|
|
|
### C5 — Encrypted CRC is not a MAC
|
|
|
|
BiPack has @CrcProtected; inspected cached source uses CRC32Source and checks at endStructure. CRC changes are predictable from a fixed-length bit-change pattern without knowing the message. For standard affine CRC conventions, the correction is CRC(delta) XOR CRC(zeros). Encrypting the CRC under a different fresh mask does not stop applying that correction. Secret CRC initialization would not fix the algebra. Keyed BLAKE2b was selected instead.
|
|
|
|
### C6 — Freshness and signature strength are different
|
|
|
|
An identity signature in HELO proves that the signer produced those bytes sometime. A captured HELO can be replayed. ACK binding the server's fresh contribution proves participation in the current handshake. This is why client authentication was moved; no claim that the signature algorithm itself was weak is intended.
|
|
|
|
### C7 — Correct interpretation of the hash-echo proposal
|
|
|
|
The assistant initially treated clientDH as public to Mallory and overstated a key-substitution attack. Correction: client session-DH is concealed in HELO. Mallory can copy the correct hash from the current response and substitute her own serverDH, so the hash does not bind the adjacent server key. But with a normal attacker-generated DH key and no access to clientDH, that attack alone does not give Mallory the shared secret or readable ACK. It causes a failed exchange, not demonstrated successful impersonation. Low-order inputs must be rejected. No complete security proof of the unsigned design was established.
|
|
|
|
Possession of the decryption private key is not inherently weaker evidence than possession of a signing private key. The distinction is what exact response/transcript is authenticated. Adding a signature was agreed, but does not automatically double security, especially if both private keys share the same compromise boundary.
|
|
|
|
### C8 — Three handshake messages versus confirmation
|
|
|
|
Three messages can provide server authentication to the client and client authentication to the server when ACK verifies. They do not by themselves tell the client that ACK arrived or authorization succeeded. A dedicated protected acceptance response was subsequently agreed in Q4. This lifecycle choice does not mean the three-message authentication argument was invalid.
|
|
|
|
## 5. Decisions and remaining specification/validation tasks
|
|
|
|
### Q1 — Server hello protection — RESOLVED
|
|
|
|
**Agreed:** derive the temporary response-encryption key with a domain-separated KDF from the fresh client session-DH public key concealed inside HELO. Send a fresh random 24-byte IV alongside the authenticated encryption of the complete server response, including its fresh session-DH key, proof and server signature. The IV is not derived deterministically from the client DH key. Exact KDF labels, cipher suite and byte encodings are specified under Q2.
|
|
|
|
```text
|
|
replyKey = KDF(concealedClientSessionDhPublicKey, replyDomain)
|
|
wire = randomIV[24] | authenticatedCiphertext(replyKey, randomIV, signedServerHello)
|
|
```
|
|
|
|
**Duplicate guard:** keep a bounded ring buffer of recently used client DH public keys and silently ignore repeats. Check/insert atomically after authenticating/decrypting HELO and validating its DH key, before allocating the new session. This is a best-effort operational guard against accidental client DH reuse, not a cryptographic security requirement. Capacity/retention are implementation parameters; eviction or restart is acceptable.
|
|
|
|
A replayed HELO alone cannot authenticate a client: the server generates fresh session-DH material, so the old ACK does not bind to the new session. Random response IVs avoid deterministic key/IV reuse when an old HELO is accepted after eviction/restart. This does not remove resource-exhaustion concerns or stop an attacker from creating fresh HELOs using the public server key; those remain Q10.
|
|
|
|
For UDP, a lost response followed by the same HELO may encounter the silent duplicate guard. A new client connection attempt therefore uses a fresh DH pair and fresh HELO; retry/lifecycle details remain Q10. Forward secrecy of client identity still comes from protecting ACK with fresh ephemeral session keys.
|
|
|
|
### Q2 — Cryptographic suite and transcript — SPECIFICATION TASK
|
|
|
|
**Direction agreed; exact specification and independent composition fixtures now exist.** This is not an outstanding architectural choice or a request for approval of every encoding detail. Reuse existing libsodium primitives, X25519/Elligator bootstrap concealment, a separate fresh session-DH exchange, Q1 response protection, server signature, session-bound client ACK and separate directional encryption/MAC keys.
|
|
|
|
Bind the handshake context as well as the connection token: both session-DH public keys and selected flavor/options. Draft exact Elligator encoding, KDF labels and inputs, session-token ordering, signed/MACed bytes, stream cipher initialization, integer encodings, failure handling, secret erasure and test vectors. Bind actual transmitted representative bytes to handle equivalent encodings. The completed specification still requires review and validation; this status does not claim those tasks are finished.
|
|
|
|
### Q3 — Server key provisioning and rotation — RESOLVED
|
|
|
|
**Application-level policy, not a communications-protocol rotation mechanism.** Each connection attempt uses fresh ephemeral keys. Longer-lived client/server identity keys and the server encryption key used for HELO are provisioned and replaced by the application. Do not add automatic rotation, overlap negotiation or key-selection machinery merely to satisfy this former question.
|
|
|
|
The server's signature authenticates its HELO response when verified against an application-trusted verifying key and bound to the agreed handshake context. A key supplied in the response alone is not a trust anchor. The protocol exposes credentials for application policy; it does not prescribe how that policy obtains, associates or rotates the keys.
|
|
|
|
### Q4 — Connected/authorized notification — RESOLVED
|
|
|
|
**Use a dedicated authenticated, session-encrypted response to the client ACK.** The server verifies the client identity proof and completes authorization before sending acceptance. The client reports connected only after verifying that acceptance.
|
|
|
|
```text
|
|
Client -> Server: encrypted HELO
|
|
Server -> Client: protected, signed server hello
|
|
Client -> Server: session-protected ACK with client identity proof
|
|
Server -> Client: session-protected acceptance
|
|
```
|
|
|
|
This is four handshake messages. Rejection, timeout and lost-message behavior still need the retry/failure details in Q10; no acceptance is sent for failed authorization.
|
|
|
|
### Q5 — RPC readiness during initialization — RESOLVED
|
|
|
|
**Reuse the original deferred readiness gate.** RPC calls made before the required identities and protected channel are ready suspend awaiting a deferred result. Once readiness is established, waiting calls may proceed through the protected RPC path. Do not send application RPC over an unprotected or identity-unready channel.
|
|
|
|
Preserve existing initialization semantics and ordering. The assistant's earlier suggestion to prohibit RPC from authorization/init handlers was not adopted; no new special pre-establishment RPC mechanism is required. Do not move deferred completion behind a handler that awaits that same deferred, creating a circular wait. Q4's protected acceptance remains the agreed client connected-notification condition.
|
|
|
|
Initialization rejection or connection failure must complete/fail the readiness wait appropriately so suspended callers are released with failure rather than left waiting indefinitely. Regression tests should cover calls queued before readiness, successful release, rejection/close, and the existing init-handler behavior. This records the intended reuse; it does not claim the future concealed-channel implementation has already passed those tests.
|
|
|
|
### Q6 — TCP framing and object authentication — RESOLVED
|
|
|
|
**Already agreed in D11:** use a continuous counter-mode/stream cipher over successive variable-length BiPack objects, each prefixed with its encoded byte length and followed by a keyed BLAKE2b authentication tag. Encrypt the length and tag as part of the same stream. No per-object IV or fixed-size chunks are introduced. Q9 subsequently adds configurable optional padding.
|
|
|
|
```text
|
|
continuousEncryption(length | protectedBody | tag | length | protectedBody | tag | ...)
|
|
protectedBody = encoded object plus optional padding, with authenticated delimitation
|
|
tag = keyedBLAKE2b(directionMacKey, sequence || exactEncodedLength || protectedBody)
|
|
```
|
|
|
|
Objects have arbitrary variable size within configured resource limits, not unlimited allocation. The receiver decrypts a bounded length, reads the bounded object and tag, verifies authentication, and only then invokes BiPack decoding/RPC dispatch. Separate directional encryption and MAC keys, implicit TCP object sequence, and a 16-byte tag remain the agreed direction. Reject authentication failure and close rather than attempting resynchronization.
|
|
|
|
Section 6 supplies the length encoding and configurable size/deadline defaults. Exact sequence encoding, counter exhaustion and test vectors remain specification/implementation work under Q2, not reasons to reopen the framing architecture. UDP retains independent records with random 24-byte IVs (D10).
|
|
|
|
### Q7 — Universal bounded-overtaking queue — RESOLVED
|
|
|
|
**Reuse the existing OutQueue algorithm for both carriers.** Its generic payload and monotonically increasing ordering value already support this. Keep the existing allowance initially (2730), with queue capacity/backpressure handled by the writer. The window prevents indefinitely admitting newer high-priority work ahead of the oldest queued item; it is not a wall-clock latency guarantee.
|
|
|
|
For TCP, assign a local admission ordinal atomically with queue insertion and store the encoded plaintext object. After pop, the single stream writer assigns the next implicit MAC sequence, computes the MAC, encrypts in actual wire order, and writes the complete record. Admission ordinals are local scheduler metadata and need not be transmitted. They are distinct from both MAC sequence numbers and stream cipher counters. Every normal queued object receives an ordinal; bypassing the bound with null ordinals would undermine the fairness guarantee.
|
|
|
|
For UDP, preserve D10: assign the protected record sequence, encrypt with a random IV and admit under the preparation mutex; the queue orders already protected records using that sequence. Thus the same scheduling algorithm applies with different preparation placement required by each carrier.
|
|
|
|
The queue bounds sender-side overtaking; it does not prevent arbitrary network reordering or loss. Partial-write/cancellation handling, counter exhaustion and tests remain implementation work under Q2/V3/V6. No additional scheduling algorithm is required.
|
|
|
|
### Q8 — Endpoint selection and optional autodetection — RESOLVED
|
|
|
|
**Use separate ports/URLs selected by application/deployment configuration.** Each listener/route knows its protocol flavor before processing the first message. No automatic detection, probing, trial parsing of legacy traffic, or client downgrade is required for the initial implementation.
|
|
|
|
Shared-endpoint autodetection is technically possible, but its rare operational benefit does not justify the initial complexity. Record it only as an optional future feature for a concrete deployment need (for example, coexistence on a single permitted TCP port); it is not a prerequisite or committed implementation task.
|
|
|
|
Dedicated endpoints do not themselves conceal the service's address or purpose. A WebSocket URL selects the route but does not hide HTTP/WebSocket setup. Bounded HELO framing and malformed-input handling are still required and belong to Q2/Q10; removing autodetection does not remove those requirements.
|
|
|
|
### Q9 — Configurable size noise — RESOLVED
|
|
|
|
**Add configurable random-length filling with random bytes to outgoing protocol records.** Apply padding inside encryption and authentication, including its delimiting/length information, so the receiver can recover the object unambiguously and an attacker cannot expose the true size by reading a padding-length field. The MAC/authentication covers the entire padded body. No unauthenticated filler is appended outside the protected record.
|
|
|
|
Expose an application configuration controlling the amount of padding, including a disabled option. Section 6 now supplies configurable starting values selected under the user's delegation; Q2 records their exact wire encoding. Keep total padded sizes within configured frame/allocation limits and the UDP datagram budget. Consider handshake records as well as application records when specifying coverage.
|
|
|
|
“Packet” here means a protocol record/message we construct, not an individual TCP segment: the operating system may split or combine writes. Padding is useful over WebSocket too: it adds uncertainty to visible message sizes even though HTTP/WebSocket presence remains recognizable. It does not hide timing, endpoints or total traffic, and repeated observations may still reveal statistical size patterns. No cover traffic or timing shaping is implied by this decision.
|
|
|
|
### Q10 — UDP handshake and resource handling — RESOLVED AT DESIGN LEVEL
|
|
|
|
Use bounded attempts and state. A new attempt always generates fresh ephemeral material and HELO, as already agreed. No automatic application-RPC retry is introduced: repeating an operation after an uncertain result is application policy.
|
|
|
|
- Lost HELO/server hello: expire the attempt and retry with a new DH pair. The recent-DH guard continues silently ignoring repeated HELOs.
|
|
- Lost client ACK or server acceptance: permit bounded retransmission of the exact protected ACK within the same attempt. Retain its authenticated identity and the exact acceptance ciphertext briefly. An identical authenticated ACK may trigger retransmission of that cached acceptance, without rerunning identity hooks, authorization, connected callbacks or application commands. Compare the exact authenticated flight/session association; do not treat arbitrary replayed data as a handshake retry. Normal data replay filtering remains unchanged.
|
|
- Do not autonomously flood acceptance messages. Resend only in response to the matching ACK and within the retry/rate budget.
|
|
- Bind incomplete UDP attempts to the receiving listener and source address/port. No transparent migration/NAT-rebinding mechanism in the initial version; reconnect with fresh keys. Do not allow unverified packets to replace an established session for that address. The detailed adapter/session-dispatch implementation must preserve this rule.
|
|
- Before return-path confirmation, bound outgoing bytes to at most three times bytes received for the attempt; there is only one server-hello response to a fresh HELO. Size/pad the legitimate HELO sufficiently for the supported response within the datagram budget. Successful decryption alone is not return-path validation: a sender knowing the public key can spoof an address. The protected ACK demonstrates receipt of the fresh server flight.
|
|
- Apply bounded admission before expensive public-key work. Use a finite worker pool without an unbounded waiting queue, per-source/global rate budgets and pending-state limits. Over-limit unauthenticated traffic is silently dropped. These controls limit resource use, not malicious access by someone able to create fresh valid HELOs.
|
|
- Initial UDP mode has one complete protected record per datagram, no new fragmentation/reassembly subsystem. Oversized messages fail locally with a clear size error; use an explicitly configured stream carrier for larger objects. Padding shrinks to the available budget before rejecting a message whose unpadded record still cannot fit. Do not silently downgrade protection.
|
|
|
|
Defaults and tunable bounds appear in section 6. UDP size guidance is informed by [RFC 8085](https://www.rfc-editor.org/rfc/rfc8085.html), which recommends avoiding fragmentation and congestion-unresponsive traffic. The selected 1200-byte default is our conservative starting budget, not a universal path-MTU guarantee or a claim that pacing alone supplies full congestion control. Retain existing transport controls; sustained Internet UDP throughput requires an appropriate congestion-control strategy and validation.
|
|
|
|
### Q11 — Closure and truncation — RESOLVED AT DESIGN LEVEL
|
|
|
|
Provide an authenticated internal CLOSE record carrying a bounded reason code. A locally requested graceful close stops new admission, drains already admitted writes up to a configurable deadline, then emits CLOSE and closes the carrier. If the deadline expires or writing fails, abort and release queued/pending operations with failure. No half-close API is introduced initially.
|
|
|
|
For TCP, a valid CLOSE at a record boundary signals clean channel termination. EOF without it is an abrupt/truncated connection; a partial record or bad MAC is a failure. None of these imply that pending RPCs succeeded. CLOSE can use a barrier after the queued records rather than bypass them through priority scheduling.
|
|
|
|
For UDP, CLOSE is best-effort, sent once; loss is possible and does not justify adding a reliable teardown protocol. Its reception means the sender closed, not that all preceding datagrams arrived. Reordering must not convert pending operations into successes. Retain existing optional liveness mechanisms; no mandatory idle heartbeat or idle-session timeout is added.
|
|
|
|
### Q12 — Integration and rollout — RESOLVED AT DESIGN LEVEL
|
|
|
|
Implement an internal protected-channel boundary around complete encoded RPC blocks. It receives explicit local priority metadata and exposes authenticated plaintext blocks. Reuse RPC correlation/dispatch and application interfaces; bypass the existing L0Call encryption wrappers for the new flavor. One scheduling owner per connection; no second priority queue that can reorder already stream-encrypted bytes.
|
|
|
|
Use separate explicit endpoint factories/configuration for the concealed flavor. Existing endpoints, public API behavior and the light TS package remain unchanged. Configuration is passed through a single immutable options object with grouped policies; reuse existing reconnection/health settings where their semantics match. No automatic fallback to the existing protocol.
|
|
|
|
During KMP prototyping, consume crypto2 0.9.6-SNAPSHOT containing the DH length guard. Do not update the currently released Kiloparsec branch merely as a documentation change. A stable concealed-protocol release must pin a qualified stable crypto2 release rather than a snapshot. Investigate the recorded JS loader failure before claiming Kotlin/JS support; the separate TypeScript client is not being changed.
|
|
|
|
### Q13 — Kotlin Elligator — IMPLEMENTATION/VALIDATION TASK
|
|
|
|
Proceed with a Kotlin prototype using existing libsodium and bignum, without adding a runtime dependency. Test mapping and shared-secret compatibility against independent vectors/implementations. Verify mixed-order input behavior through the actual platform bindings. Treat timing suitability as a dataflow and threat-model question, not something resolved by arithmetic tests alone. The [timing assessment](CONCEALED-PROTOCOL-TIMING.md) records the current public-coordinate dataflow, JVM/Linux Native measurements and the proposed 50 ms roundup experiment; variable execution time is not itself evidence of private-key leakage.
|
|
|
|
Use fixed test vectors and a fixed-seed randomized corpus (at least 10,000 round trips in the extended suite), boundary/exceptional cases and malformed inputs. Test mask/sign/padding choices and low-order rejection. Measure timing, allocations and retained binary size on JVM and Linux Native. Statistical distinguishers detect regressions; they do not prove cryptographic indistinguishability or constant-time execution.
|
|
|
|
If these investigations show general bignum unsuitable, implement reviewed fixed-size Kotlin field arithmetic and validate it before production. This stays within the no-new-runtime-dependency direction. Any later decision to add a native library or weaken a security property requires a new design decision; do not silently do either.
|
|
|
|
|
|
**2026-10-22 user decision: bounded HELO failure work.** Each shared server identity keeps an independent random throwaway X25519 scalar seed. Only a failed HELO envelope authentication triggers one extra decryption attempt, using a temporary seed copy with 1–8 distinct effective scalar bits flipped. Discard every dummy outcome; preserve the original authentication rejection. Do not modify the real key, send a decoy response, invoke identity hooks, recursively retry or admit the peer. Erase the temporary key copy after use. Keep the attempt within existing crypto admission bounds. `ConcealedOptions.Handshake.decoyOnHelloAuthenticationFailure` defaults to `true` and can disable this local policy. No wire format change, clock-padding policy or random division loop is introduced. The [timing assessment, section 7](CONCEALED-PROTOCOL-TIMING.md#7-requested-server-side-decoy-decryption) records tests and measured cost. This is bounded failure-path work, without a claim that key mutation defeats every timing or microarchitectural attack.
|
|
|
|
|
|
### Q14 — Qualification — IMPLEMENTATION/VALIDATION TASK
|
|
|
|
Keep the new flavor experimental until the wire specification and V1–V7 checks pass. First qualify JVM and Linux Native; compile other configured KMP targets and label runtime support according to evidence. Do not advertise browser/mobile/Apple runtime qualification based only on compilation. Leave the current protocol available throughout rollout.
|
|
|
|
Use deterministic wire vectors across implementations, negative handshake/MAC tests, bounded-input fuzzing, real loopback UDP impairment, TCP segmentation/coalescing, concurrent duplex and fairness tests. Extended CI starts with 10,000 deterministic malformed-input cases and a 60-second bounded duplex/priority run; soak durations and seeds are test-harness settings, not wire parameters. Expand these workloads based on findings rather than treating their counts as a security proof.
|
|
|
|
Before calling this a production concealed protocol, review the full handshake and custom stream-plus-MAC construction independently of the implementation, including the KDF from concealed clientDH, identity/session binding, rejection behavior, resource limits, truncation and forward-secrecy claims. This review is work still to do, not a completed audit. No further user design question is needed to begin the specification/prototype.
|
|
|
|
## 6. Configurable implementation defaults
|
|
|
|
These are assistant-selected starting defaults under the user's instruction to settle routine choices. They are not protocol standards or measured optimal values. Keep them in one immutable `ConcealedConnectionOptions` with grouped `handshake`, `limits`, `padding`, `queue`, `udp` and `close` policies (names provisional). Validate ranges at construction and expose application overrides. Do not change existing-flavor defaults.
|
|
|
|
| Setting | Initial default | Semantics |
|
|
| --- | --- | --- |
|
|
| Handshake attempt deadline | 15 seconds | Whole attempt, including hooks; override for interactive/external authorization. |
|
|
| Maximum connection attempts | 3 | Includes first attempt; finite by default, fresh DH/HELO each time. |
|
|
| Delay between fresh attempts | 700 ms, exponential backoff capped at 5 s, ±20% jitter | Align base delay with existing reconnection configuration; cancellation interrupts waits. |
|
|
| UDP ACK retry interval | 1 s, doubling to 4 s, ±20% jitter | Only after a valid server hello; at most 3 retransmissions and never past attempt deadline. |
|
|
| Incomplete server state lifetime | 15 seconds | Starts at admission, not reset indefinitely by duplicate traffic. |
|
|
| Cached acceptance lifetime | 30 seconds | Expire cryptographic retry state independently of established session lifetime. Bound cache entries. |
|
|
| Recent-DH guard | 4096 entries | Optional; 0 disables. Ring eviction, no persistence requirement. Operational only. |
|
|
| Pending handshakes | 1024 global / 8 per source IP | Per-listener starting budgets; shared-NAT deployments may raise the per-source limit. |
|
|
| Acceptance retry cache | One bounded entry per active server connection | Lifetime configurable; total bounded by listener carrier limits. No independent unbounded map. |
|
|
| Bootstrap crypto concurrency | 4 per listener | Reject excess before expensive work; configurable for server CPU capacity. |
|
|
| New-HELO admission | 100/s global, burst 200; 10/s per source IP, burst 20 | Bounded token buckets; expire inactive source entries and cap source table at 4096 entries. Reject new entries when full rather than allowing unlimited allocation. |
|
|
| Maximum UDP datagram payload | 1200 bytes | Includes all protocol/crypto overhead and padding; configurable for known paths, obey platform/protocol absolute bounds. No claim of universal MTU safety. |
|
|
| Maximum bootstrap frame on a stream carrier | 16 KiB | Bound first-message parsing before a session exists. UDP uses its stricter datagram limit. |
|
|
| Maximum protected stream body | 16 MiB | Includes inner object delimitation and padding, excludes fixed outer length/tag; checked before allocation. |
|
|
| Length encoding | Fixed unsigned 32-bit, big-endian | Encrypted 4-byte outer body length; authenticated inner object byte length before object+padding. Avoid unbounded varint parsing. |
|
|
| Partial stream-record deadline | 30 seconds | Starts when the first byte of a new record arrives; does not time out an otherwise idle connection. Configurable for slow links/large records. |
|
|
| Outgoing queue | 128 items / 32 MiB | Reuse current writer defaults; enforce byte budget including protection/padding cost. |
|
|
| Maximum priority overtaking | 2730 admission positions | Reuse OutQueue behavior; all regular entries numbered. |
|
|
| UDP replay window | 4096 sequence positions | Existing baseline; local overtaking <= floor(2*window/3). |
|
|
| Random padding | Uniform 0–255 bytes per record | Fresh random length and bytes, protected with body. Explicit disabled option; effective maximum capped by carrier/record/queue budget. Handshake minimum padding for response budget is separate. |
|
|
| Graceful-close drain deadline | 2 seconds | Abort if drain/close cannot finish; no false success for unanswered RPCs. |
|
|
| Heartbeat | Disabled unless application enables existing health policy | Do not add idle polling as part of concealment. |
|
|
|
|
Padding layout for stream records becomes `encrypted(outerBodyLength | innerObjectLength | objectBytes | padding | tag)`, with the tag covering the implicit sequence, exact outer length and entire body. Verify before interpreting inner length or decoding the object. Outer length is fixed-width but still untrusted until authentication; cap the read and deadline. The same protected inner object-length/body convention can be used in independent UDP records, alongside their encrypted sequence number. Specify exact control/message types in Q2.
|
|
|
|
Local receive limits and any peer-advertised limits must be reconciled in the authenticated handshake (use the smaller applicable bound); never allocate from a peer claim beyond the local cap. The bootstrap starts with bounded endpoint-configured limits before negotiation. Do not silently change wire encodings or crypto algorithms through ordinary tuning parameters.
|
|
|
|
The 16-byte MAC tag, 24-byte UDP IV, key separation, identity verification, DH-error rejection, authenticated-before-dispatch rule and counter non-wrap are fixed security properties, not options to disable. For the first wire version, keep the replay window at 4096 and validate configurable overtaking against it; changing the window requires explicit authenticated negotiation/specification support. Specify cipher usage ceilings/rekeying in Q2; fail closed before limits and reconnect with fresh keys rather than reset counters under an old key.
|
|
|
|
## 7. Proposed validation sequence
|
|
|
|
1. **V1 — Elligator compatibility:** reference vectors; encode/decode; sign/top-bit choices; exceptional inputs; randomized subgroup coverage; agreement between Kotlin/libsodium and an independent implementation.
|
|
2. **V2 — Bootstrap/handshake:** mutation of every authenticated field, replayed HELO/ACK, substituted DH, wrong server identity, rejected client authorization, transcript/role confusion, all-zero DH rejection, safe failure behavior.
|
|
3. **V3 — TCP:** arbitrary segmentation/coalescing and partial cipher-mask blocks; authenticated length mutation; truncation; MAC changes; deadlines; size limits; no object deserialization/dispatch before verification; cancellation mid-write.
|
|
4. **V4 — UDP:** duplicate, stale, reordered, lost and replayed records; independent decryption; bounded handshake retransmission/cache behavior; nonce/sequence exhaustion handling.
|
|
5. **V5 — RPC/lifecycle:** duplex calls, responses, errors, pushes, initialization RPCs, authorization rejection, connection notification and close semantics.
|
|
6. **V6 — Scheduling/performance:** sustained priorities with low-priority progress, large-message latency, bounded queues, concurrent producers, allocation/cipher/MAC cost and actual artifact-size delta.
|
|
7. **V7 — Platform/security review:** JVM and Linux Native first, then the explicitly supported KMP targets; reference implementation outside production dependencies; randomness distinguishers as regression tests, not proofs; review the custom stream-plus-MAC composition before release.
|
|
|
|
## 8. Sources consulted
|
|
|
|
- [Elligator paper](https://elligator.cr.yp.to/elligator-20130828.pdf)
|
|
- [Elligator key-exchange construction](https://elligator.org/key-exchange)
|
|
- [Elligator direct/inverse maps](https://elligator.org/map)
|
|
- [Monocypher Elligator manual](https://monocypher.org/manual/elligator) — reference, not chosen runtime dependency.
|
|
- [Monocypher security disclosures](https://monocypher.org/quality-assurance/disclosures) — use a reference version including applicable timing fixes.
|
|
- [Libsodium point arithmetic](https://libsodium.gitbook.io/doc/advanced/point-arithmetic)
|
|
- [Libsodium sealed boxes](https://doc.libsodium.org/public-key_cryptography/sealed_boxes)
|
|
- [Libsodium key exchange](https://doc.libsodium.org/key_exchange)
|
|
- [Libsodium stream ciphers](https://doc.libsodium.org/advanced/stream_ciphers)
|
|
- [Libsodium secretstream](https://doc.libsodium.org/secret-key_cryptography/secretstream) — investigated alternative; still requires ciphertext chunk boundaries, not selected for the current TCP plan.
|
|
- [RFC 7748](https://www.rfc-editor.org/rfc/rfc7748.html) — X25519 encoding and DH result checking.
|
|
- [RFC 8017](https://www.rfc-editor.org/rfc/rfc8017.html) — RSA ciphertext range.
|
|
- [Ionspin KMP bignum](https://github.com/ionspin/kotlin-multiplatform-bignum)
|
|
- [RFC 8085: UDP usage guidelines](https://www.rfc-editor.org/rfc/rfc8085.html) — datagram sizing, reliability and congestion considerations.
|
|
|
|
An experimental source implementation and wire draft now exist; see the implementation record for completed validation and remaining qualification. The design below preserves the decisions and rationale, not a production-security claim. Q1/Q3–Q12 are resolved at design level. Q2 is the wire-specification task; Q13/Q14 and V1–V7 are implementation/validation work, not completed security claims. The concrete specification, prototype and expanded validation now exist; the [qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md) is the current continuation point. Retain the stable identifiers.
|