Release 0.9.0-SNAPSHOT with concealed KMP transports

This commit is contained in:
sergeych committed 2026-10-22 02:41:08 +04:00
1 parent 009d62a155
commit 73a785d45c
55 files changed
+6051 -25

No files matched your search

+463
View File
@@ -0,0 +1,463 @@
# Concealed Kiloparsec protocol: design record
Recorded: 2026-10-21; implementation work started 2026-10-22. Current candidate: 0.9.0-SNAPSHOT, an experimental source implementation and wire draft with [qualification evidence](CONCEALED-PROTOCOL-QUALIFICATION.md); not an audited or universally runtime-qualified protocol.
See [wire specification](CONCEALED-PROTOCOL-WIRE.md), [implementation/continuation](CONCEALED-PROTOCOL-IMPLEMENTATION.md) and the [timing assessment](CONCEALED-PROTOCOL-TIMING.md) for current code, usage, measurements and validation status.
This document preserves the conversation before further design work. It distinguishes agreed direction from assistant proposals and unresolved details. It is the continuation point for this work. The existing protocol-2 review is [REVIEW-237d404.md](REVIEW-237d404.md); its issue numbers I1–I7 concern the released implementation, not this new design.
Stable references: **D1–D14** are decisions/directions, **C1–C8** are corrections and reasoning, **Q1–Q14** are discussion questions (Q1/Q3–Q12 resolved at design level; Q2 is a specification task; Q13/Q14 are implementation/validation tasks), **V1–V7** are proposed validation steps. Do not renumber existing references; append new ones. Questions are not approval gates for routine work. Discuss them individually before freezing the wire format.
## 1. Objective and scope
### D1 — Conceal protocol metadata as well as application data
The present protocol has recognizable envelopes and exposed routing metadata. The new flavor should make its wire bytes look random, including the initial handshake, while authenticating the information needed to process records. No visible magic prefix, protocol version, command name, RPC identifier, priority, error code or sequential packet counter should be required in the concealed encoding.
This is a second connection flavor, not an instruction to replace the released protocol. Use separate configured ports/URLs for the existing and concealed flavors (resolved Q8). Endpoint configuration selects the protocol before parsing the handshake; no wire marker or autodetection is required. The original shared-endpoint autodetection idea is deferred as an optional future capability, not initial scope.
Random-looking bytes do not conceal endpoints, traffic volume, timing, packet lengths or application traffic patterns. A raw TCP connection and a WebSocket connection also expose different carrier information. No claim of universal undetectability, censorship resistance, or security against all active attacks has been established.
### D2 — KMP first, no new crypto library dependency
Scope is the Kotlin Multiplatform library. Leave the lightweight TypeScript client and its present protocol behavior unchanged. A later TS implementation is possible, but is not part of the initial implementation.
Reuse the existing libsodium bindings and Ionspin KMP bignum library. Avoid adding Monocypher or another runtime crypto library solely for Elligator. Independent implementations may be used as test references without becoming shipped dependencies.
### D3 — A pre-known, publicly distributable server encryption key
The client knows a suitable server encryption public key before the handshake. It may be obtained from a site, API, an earlier authenticated connection, or another provisioning mechanism. Confidential distribution is not required. Authentic distribution is still necessary to know which server is being authenticated.
A shared confidential entrance secret was proposed and not selected. Anyone knowing the public encryption key can construct a new encrypted HELO. This means the design does not prevent active probes by such parties merely through bootstrap encryption.
Separate encryption and signing keys are involved if we use both proofs. Their authentic association, provisioning and replacement belong to application policy (resolved Q3); automatic rotation is outside the communications protocol.
## 2. Cryptographic bootstrap
### D4 — X25519 with Elligator 2 concealment
RSA was considered, but not selected. Ordinary X25519 ephemeral key encodings are distinguishable from uniformly random 32-byte strings. Elligator 2 is the selected direction for concealing the exposed ephemeral bootstrap key.
Conceptual bootstrap layout, not yet a wire specification:
```text
32-byte Elligator representative | authentication tag | encrypted HELO
```
The discussion used a 16-byte tag and the 48-byte overhead of libsodium sealed boxes as the model. This does not mean an unchanged crypto_box_seal call implements the intended bootstrap. Exact suite, derivation encodings and authenticated data remain Q2; the response-protection choice is resolved in Q1.
The server's static encryption public key need not itself be Elligator-encoded in the packet: it is pre-known. Only ephemeral key material exposed on the wire needs concealment. Subsequent DH keys can be carried inside encryption.
Correct concealed key generation requires more than applying an inverse map to an ordinary public key:
1. Generate a fresh ephemeral secret.
2. Generate the specially randomized public point, including the required small-order component/sign handling.
3. Apply the inverse Elligator map; regenerate locally if the point is not representable.
4. Encode with the required random choices, including otherwise unused high bits.
Approximately two generation attempts are expected for a representable point. This is distinct from rejecting an invalid peer DH input: never loop regenerating our key to accommodate a malicious low-order peer key.
### D5 — Implement the mapping in Kotlin using existing arithmetic
Verified dependency graph in Kiloparsec:
```text
crypto2:0.9.0 -> com.ionspin.kotlin:bignum:0.3.10
```
BigInteger/ModularBigInteger, not BigDecimal, supply the relevant arithmetic. Existing bindings 0.9.6 expose Edwards base multiplication, point addition and X25519. Proposed split:
| Work | Component |
| --- | --- |
| Random bytes, base multiplication, point addition, X25519 | Existing libsodium |
| Edwards/Montgomery conversion, Elligator maps, encoding | Kotlin + existing bignum |
| Hashing, authentication and encryption | Existing libsodium |
Libsodium's scalar arithmetic is modulo the subgroup order; Elligator field arithmetic is modulo p = 2^255 - 19. They are not interchangeable. Its from_uniform function maps to a subgroup point; it does not provide the inverse concealed-key generation we need.
This is a feasible direction, not a tested implementation. Mixed-order point acceptance must be checked through the actual bindings. General bignum arithmetic is not assumed constant-time. Concealed intermediate points cannot automatically be treated as harmless public information. Start with a readable, independently checked prototype, then evaluate timing and production suitability. Added binary size must be measured: using already-present dependencies can retain additional previously stripped code.
### D6 — Move client identity proof into a fresh-session ACK
Agreed handshake direction:
```text
C -> S: HELO, encrypted to the known server encryption key
fresh client session-DH public key; no client identity/signature
S -> C: server hello
fresh server session-DH public key
proof relating to this HELO
server signature (agreed addition)
encrypted/authenticated with the Q1 reply key and a fresh random 24-byte IV
C -> S: ACK, encrypted/authenticated under fresh session keys
client identity public key
client signature binding this session
S -> C: acceptance, encrypted/authenticated under fresh session keys
sent after ACK verification and successful application authorization
```
Both session-DH key pairs are fresh for each connection attempt. Remove the client identity public key as well as its signature from HELO. Moving only the signature would not conceal identity against later compromise of the server's static decryption key.
The user proposed the connection token derived from the directional session-key pair in the proper order as the ACK's signed value. The current SafeKeyExchange session tag is a relevant existing mechanism, but the exact new token derivation and signature input are not frozen. Role/domain separation and handshake-context binding were agreed: cover the connection token, both session-DH public keys and selected flavor/options. Q2 remains the task of specifying exact inputs and encodings.
Once ACK arrives and verifies, the server has authenticated the client. It then completes application authorization and sends a protected acceptance response. The client reports connected only after verifying this response. This fourth handshake message is agreed (resolved Q4).
### D7 — Server proof and signature
The user proposed BLAKE2b(clientSessionDhPublicKey) as proof that the server decrypted HELO. That client public key is fresh and concealed inside HELO; it is not necessarily the exposed ephemeral bootstrap key. Under that assumption, the hash is evidence of knowing the concealed value.
The user subsequently agreed to add a server signature over the server DH key and that proof. The exact signed transcript, role label, options and identity-key association remain Q2/Q3.
Q1 is resolved: derive a domain-separated temporary reply key from the fresh client session-DH public key concealed inside HELO. Encrypt/authenticate the complete signed server response using that key and a fresh random 24-byte IV, transmitted alongside the ciphertext. This uses the concealed client DH value as bootstrap material, not the exposed Elligator representative. Retain the agreed server signature. Exact suite and derivation encodings remain Q2.
### D8 — Reject failed DH agreement at the crypto interface
Libsodium crypto_kx already rejects unacceptable low-order inputs before deriving session keys. Inspected JVM and Linux bindings propagate failures. Runtime tests through crypto2 verified rejection on both roles, including nonzero low-order encodings and high-bit aliases.
The check concerns failure of the operation / an all-zero raw DH result, not merely whether received public-key bytes contain a nonzero byte. Checking derived session keys for zero is not a substitute: those keys are hashes of the DH result and context.
The crypto2 investigation found a separate missing length guard. SafeKeyExchange now rejects peer public-key arrays whose length is not exactly 32 before entering native code. This was published as crypto2 0.9.6-SNAPSHOT to Maven Local/private Maven and committed/pushed as 70d7a13. All 67 JVM and 67 Linux Native tests passed. Its focused JS tests fail during the existing libsodium loader initialization (_sodium_init is not a function), not DH validation. Other runtimes were not qualified.
Kiloparsec has not been switched from its existing crypto2 dependency as part of this design discussion.
## 3. Established connection
### D9 — One protection layer below RPC
Do not tunnel the current encrypted L0Call protocol inside the new encrypted channel. Serialize ordinary RPC blocks, then protect them once at the channel layer.
```text
Command / RemoteInterface / LocalInterface / application scope
-> RPC calls, responses, errors, pushes and correlation
-> protected channel
-> carrier
```
Reuse application APIs and authorization hooks. A serialized Transport.Block can initially serve as the inner message. Its IDs, names, priority and error metadata are then entirely concealed and authenticated. Avoid duplicating those fields in another ProtectedMessage merely to preserve the old wrapping architecture.
Current relevant implementation points:
- Transport.kt: block serialization, correlation, pending waiters, dispatch and sending.
- KiloRemoteInterface.kt and KiloL0Interface.kt: current L0Call/protected-message wrapping.
- KiloClientConnection.kt and KiloServerConnection.kt: current handshake and lifecycle.
- KiloParams.kt: current session cipher/counter handling.
- ReplayWindow.kt, adapter/OutQueue.kt, adapter/PriorityBlockWriter.kt: replay/scheduling machinery.
The exact common interface and refactoring boundary remain Q12. Device currently carries bytes and is not, by itself, a sufficient interface for carrying local scheduling metadata.
### D10 — UDP: independent records with random 24-byte IVs
Agreed for the datagram path:
```text
random IV [24 bytes] | authentication tag [16 bytes] | ciphertext {
sequence number | RPC block | optional padding
}
```
The IV is the encryption nonce. Generate it with the CSPRNG; no session-seeded custom PRNG construction is selected. Random generation gives negligible collision probability, not strict mathematical uniqueness. For q independent 192-bit nonces under one directional key, collision probability is approximately q(q-1)/2^193.
Fresh directional session keys isolate the two directions. The concealed sequential counter performs replay tracking and queue ordering; the random IV permits independent decryption after loss/reordering. Authenticate before updating replay state, dispatching a command or consuming a pending response waiter. Duplicate/stale records are nonfatal.
Retain the preparation mutex and sequence-aware priority queue: allocate sequence, encrypt and admit atomically; await actual transmission outside the mutex. Counter exhaustion must not wrap. The existing 4096-position window and 2730 maximum local overtaking are the reuse baseline, not a guarantee against arbitrary network delay.
Q9 adds configurable random-length random-byte padding inside the protected record. No packet-loss recovery for application RPCs is implicitly added by encryption or replay tracking.
### D11 — TCP: continuous encryption plus object authentication
The user chose a continuous counter-mode/stream-cipher construction. Each direction maintains its own keystream position; arbitrary socket write/read segmentation must not affect it. Use fresh directional encryption keys and a defined initial nonce/counter; never restart the stream under reused key/nonce material.
Agreed conceptual plaintext stream:
```text
length | encoded object + optional padding | keyed BLAKE2b tag
length | encoded object + optional padding | keyed BLAKE2b tag
...
```
Encrypt every byte, including lengths and tags, with the continuous stream. No per-object IV is needed on this path. TCP's ordered delivery synchronizes the stream position. This supersedes the random-per-record-IV proposal for TCP only; UDP retains D10.
Proposed MAC input, accepted conceptually:
```text
tag = keyedBLAKE2b(macKey, sequence || exactEncodedLength || exactProtectedBodyBytes)
```
The protected body contains the encoded object and optional Q9 padding, with authenticated delimitation between them. Exact padded-body length encoding is Q2 specification work. The MAC covers the entire body, including padding and its delimiters. Use a separately derived MAC key for each direction and a 16-byte tag. The TCP object sequence may be implicit, incrementing on authenticated objects. Exact integer encodings, domain separation, cipher and limits remain Q2/Q6/Q7.
Receive pipeline:
1. Decrypt a bounded length header.
2. Reject invalid/over-limit lengths before allocation or body reading.
3. Read bounded object bytes and tag with bounded waiting.
4. Verify the MAC in constant time.
5. Only then decode BiPack and invoke RPC/application logic.
The length remains untrusted until MAC verification. A modified length can cause a bounded read, wait or connection failure; the construction does not eliminate denial of service. On invalid MAC, abort; do not scan for resynchronization or continue consuming objects.
This is a custom construction requiring validation, not an already-audited equivalent of a standard AEAD record protocol. Authenticate the exact received bytes, not a decoded-and-reserialized object. Authenticated clean closure is selected in resolved Q11.
### D12 — Priority and fairness must survive the new layering
The current queue's bounded nonce overtaking intrinsically prevents indefinite postponement of old messages. The user explicitly valued that property over a simpler lambda queue that encrypts only when popped.
TCP's continuous cipher must be advanced in actual wire order. Therefore select complete encoded objects before advancing stream encryption; serialize writes through one writer. Once bytes of an object enter the stream, another object's bytes cannot be arbitrarily interleaved with them.
Resolved Q7: reuse the existing generic OutQueue bounded-overtaking algorithm for TCP as well as UDP. For TCP, its ordering field is a monotonically increasing local admission ordinal, not the stream cipher position or implicit MAC sequence. Allocate the ordinal and enqueue atomically. Queue encoded plaintext objects; a single writer pops by priority within the bounded window, assigns the wire-order MAC sequence, authenticates/encrypts, and writes. For UDP, retain the sequence-aware ciphertext queue from D10. A large in-progress object can delay small urgent objects; cancellation after writing starts must finish that record or close the stream.
### D13 — Preserve application authorization and scope
Kotlin intentionally does not reduce server/client trust policy to a constructor list of keys. Verified remote credentials and session information must be exposed through application scope/init handlers. SecurityException and BreakConnectionException subclasses must still abort initialization.
Preserve the existing deferred readiness mechanism (resolved Q5): calls made before readiness suspend awaiting the result. RPC starts only when identities are ready and the channel is protected. Preserve existing initialization/application-policy ordering rather than introducing a new pre-establishment RPC mechanism or a blanket prohibition on RPC from init handlers. Application authorization remains enforced by its existing hooks/policy; Q4 governs the client connected notification after protected acceptance.
Protocol replay prevention is not business-operation deduplication after reconnect. A new valid request may be an intentional repeat; application idempotency remains application responsibility.
### D14 — Compatibility and release scope
Current Kiloparsec 0.8.0 was published and pushed as 009d62a / tag v0.8.0. The review's I1–I5 repairs are included; I6/I7 were closed as intentional decisions/non-defects. The concealed protocol is future work and must not be represented as part of that release.
No new wire version number, public API name or release version has been chosen. Avoid calling the new flavor "protocol 2" ambiguously: the existing released authenticated envelope already uses protocol 2.
## 4. Corrections that must not be lost
### C1 — EC encryption is hybrid encryption
X25519's 32 bytes are a public coordinate/shared-secret size, not an RSA-style plaintext block size. Public-key encryption normally exposes an ephemeral public key plus symmetric ciphertext/tag. The ephemeral bootstrap key and the client session-DH key concealed inside HELO are separate in the discussed design; their reuse/separation must be explicit in Q2.
### C2 — Ordinary public keys leave a statistical fingerprint
For normally generated X25519 encodings, canonical field range, curve membership and prime-subgroup membership give public tests. Roughly 1/32 of uniformly random 32-byte strings passes all three. This is a distinguisher against uniform bytes, not a probability that a packet is Kiloparsec. Multiple independent observations strengthen it. Poly1305 tags have no analogous public validity test under the usual assumptions.
### C3 — RSA does not automatically look uniform either
A ciphertext integer is less than the public modulus n. Random k-bit values pass that range test with probability n/2^k. RSA was not adopted as an escape from encoding distinguishers.
### C4 — Continuous XOR encryption does not authenticate
Fresh masks and fresh keys do not prevent chosen bit flips in a single ciphertext: (P XOR K XOR delta) XOR K = P XOR delta. The attack does not require keystream reuse.
### C5 — Encrypted CRC is not a MAC
BiPack has @CrcProtected; inspected cached source uses CRC32Source and checks at endStructure. CRC changes are predictable from a fixed-length bit-change pattern without knowing the message. For standard affine CRC conventions, the correction is CRC(delta) XOR CRC(zeros). Encrypting the CRC under a different fresh mask does not stop applying that correction. Secret CRC initialization would not fix the algebra. Keyed BLAKE2b was selected instead.
### C6 — Freshness and signature strength are different
An identity signature in HELO proves that the signer produced those bytes sometime. A captured HELO can be replayed. ACK binding the server's fresh contribution proves participation in the current handshake. This is why client authentication was moved; no claim that the signature algorithm itself was weak is intended.
### C7 — Correct interpretation of the hash-echo proposal
The assistant initially treated clientDH as public to Mallory and overstated a key-substitution attack. Correction: client session-DH is concealed in HELO. Mallory can copy the correct hash from the current response and substitute her own serverDH, so the hash does not bind the adjacent server key. But with a normal attacker-generated DH key and no access to clientDH, that attack alone does not give Mallory the shared secret or readable ACK. It causes a failed exchange, not demonstrated successful impersonation. Low-order inputs must be rejected. No complete security proof of the unsigned design was established.
Possession of the decryption private key is not inherently weaker evidence than possession of a signing private key. The distinction is what exact response/transcript is authenticated. Adding a signature was agreed, but does not automatically double security, especially if both private keys share the same compromise boundary.
### C8 — Three handshake messages versus confirmation
Three messages can provide server authentication to the client and client authentication to the server when ACK verifies. They do not by themselves tell the client that ACK arrived or authorization succeeded. A dedicated protected acceptance response was subsequently agreed in Q4. This lifecycle choice does not mean the three-message authentication argument was invalid.
## 5. Decisions and remaining specification/validation tasks
### Q1 — Server hello protection — RESOLVED
**Agreed:** derive the temporary response-encryption key with a domain-separated KDF from the fresh client session-DH public key concealed inside HELO. Send a fresh random 24-byte IV alongside the authenticated encryption of the complete server response, including its fresh session-DH key, proof and server signature. The IV is not derived deterministically from the client DH key. Exact KDF labels, cipher suite and byte encodings are specified under Q2.
```text
replyKey = KDF(concealedClientSessionDhPublicKey, replyDomain)
wire = randomIV[24] | authenticatedCiphertext(replyKey, randomIV, signedServerHello)
```
**Duplicate guard:** keep a bounded ring buffer of recently used client DH public keys and silently ignore repeats. Check/insert atomically after authenticating/decrypting HELO and validating its DH key, before allocating the new session. This is a best-effort operational guard against accidental client DH reuse, not a cryptographic security requirement. Capacity/retention are implementation parameters; eviction or restart is acceptable.
A replayed HELO alone cannot authenticate a client: the server generates fresh session-DH material, so the old ACK does not bind to the new session. Random response IVs avoid deterministic key/IV reuse when an old HELO is accepted after eviction/restart. This does not remove resource-exhaustion concerns or stop an attacker from creating fresh HELOs using the public server key; those remain Q10.
For UDP, a lost response followed by the same HELO may encounter the silent duplicate guard. A new client connection attempt therefore uses a fresh DH pair and fresh HELO; retry/lifecycle details remain Q10. Forward secrecy of client identity still comes from protecting ACK with fresh ephemeral session keys.
### Q2 — Cryptographic suite and transcript — SPECIFICATION TASK
**Direction agreed; exact specification and independent composition fixtures now exist.** This is not an outstanding architectural choice or a request for approval of every encoding detail. Reuse existing libsodium primitives, X25519/Elligator bootstrap concealment, a separate fresh session-DH exchange, Q1 response protection, server signature, session-bound client ACK and separate directional encryption/MAC keys.
Bind the handshake context as well as the connection token: both session-DH public keys and selected flavor/options. Draft exact Elligator encoding, KDF labels and inputs, session-token ordering, signed/MACed bytes, stream cipher initialization, integer encodings, failure handling, secret erasure and test vectors. Bind actual transmitted representative bytes to handle equivalent encodings. The completed specification still requires review and validation; this status does not claim those tasks are finished.
### Q3 — Server key provisioning and rotation — RESOLVED
**Application-level policy, not a communications-protocol rotation mechanism.** Each connection attempt uses fresh ephemeral keys. Longer-lived client/server identity keys and the server encryption key used for HELO are provisioned and replaced by the application. Do not add automatic rotation, overlap negotiation or key-selection machinery merely to satisfy this former question.
The server's signature authenticates its HELO response when verified against an application-trusted verifying key and bound to the agreed handshake context. A key supplied in the response alone is not a trust anchor. The protocol exposes credentials for application policy; it does not prescribe how that policy obtains, associates or rotates the keys.
### Q4 — Connected/authorized notification — RESOLVED
**Use a dedicated authenticated, session-encrypted response to the client ACK.** The server verifies the client identity proof and completes authorization before sending acceptance. The client reports connected only after verifying that acceptance.
```text
Client -> Server: encrypted HELO
Server -> Client: protected, signed server hello
Client -> Server: session-protected ACK with client identity proof
Server -> Client: session-protected acceptance
```
This is four handshake messages. Rejection, timeout and lost-message behavior still need the retry/failure details in Q10; no acceptance is sent for failed authorization.
### Q5 — RPC readiness during initialization — RESOLVED
**Reuse the original deferred readiness gate.** RPC calls made before the required identities and protected channel are ready suspend awaiting a deferred result. Once readiness is established, waiting calls may proceed through the protected RPC path. Do not send application RPC over an unprotected or identity-unready channel.
Preserve existing initialization semantics and ordering. The assistant's earlier suggestion to prohibit RPC from authorization/init handlers was not adopted; no new special pre-establishment RPC mechanism is required. Do not move deferred completion behind a handler that awaits that same deferred, creating a circular wait. Q4's protected acceptance remains the agreed client connected-notification condition.
Initialization rejection or connection failure must complete/fail the readiness wait appropriately so suspended callers are released with failure rather than left waiting indefinitely. Regression tests should cover calls queued before readiness, successful release, rejection/close, and the existing init-handler behavior. This records the intended reuse; it does not claim the future concealed-channel implementation has already passed those tests.
### Q6 — TCP framing and object authentication — RESOLVED
**Already agreed in D11:** use a continuous counter-mode/stream cipher over successive variable-length BiPack objects, each prefixed with its encoded byte length and followed by a keyed BLAKE2b authentication tag. Encrypt the length and tag as part of the same stream. No per-object IV or fixed-size chunks are introduced. Q9 subsequently adds configurable optional padding.
```text
continuousEncryption(length | protectedBody | tag | length | protectedBody | tag | ...)
protectedBody = encoded object plus optional padding, with authenticated delimitation
tag = keyedBLAKE2b(directionMacKey, sequence || exactEncodedLength || protectedBody)
```
Objects have arbitrary variable size within configured resource limits, not unlimited allocation. The receiver decrypts a bounded length, reads the bounded object and tag, verifies authentication, and only then invokes BiPack decoding/RPC dispatch. Separate directional encryption and MAC keys, implicit TCP object sequence, and a 16-byte tag remain the agreed direction. Reject authentication failure and close rather than attempting resynchronization.
Section 6 supplies the length encoding and configurable size/deadline defaults. Exact sequence encoding, counter exhaustion and test vectors remain specification/implementation work under Q2, not reasons to reopen the framing architecture. UDP retains independent records with random 24-byte IVs (D10).
### Q7 — Universal bounded-overtaking queue — RESOLVED
**Reuse the existing OutQueue algorithm for both carriers.** Its generic payload and monotonically increasing ordering value already support this. Keep the existing allowance initially (2730), with queue capacity/backpressure handled by the writer. The window prevents indefinitely admitting newer high-priority work ahead of the oldest queued item; it is not a wall-clock latency guarantee.
For TCP, assign a local admission ordinal atomically with queue insertion and store the encoded plaintext object. After pop, the single stream writer assigns the next implicit MAC sequence, computes the MAC, encrypts in actual wire order, and writes the complete record. Admission ordinals are local scheduler metadata and need not be transmitted. They are distinct from both MAC sequence numbers and stream cipher counters. Every normal queued object receives an ordinal; bypassing the bound with null ordinals would undermine the fairness guarantee.
For UDP, preserve D10: assign the protected record sequence, encrypt with a random IV and admit under the preparation mutex; the queue orders already protected records using that sequence. Thus the same scheduling algorithm applies with different preparation placement required by each carrier.
The queue bounds sender-side overtaking; it does not prevent arbitrary network reordering or loss. Partial-write/cancellation handling, counter exhaustion and tests remain implementation work under Q2/V3/V6. No additional scheduling algorithm is required.
### Q8 — Endpoint selection and optional autodetection — RESOLVED
**Use separate ports/URLs selected by application/deployment configuration.** Each listener/route knows its protocol flavor before processing the first message. No automatic detection, probing, trial parsing of legacy traffic, or client downgrade is required for the initial implementation.
Shared-endpoint autodetection is technically possible, but its rare operational benefit does not justify the initial complexity. Record it only as an optional future feature for a concrete deployment need (for example, coexistence on a single permitted TCP port); it is not a prerequisite or committed implementation task.
Dedicated endpoints do not themselves conceal the service's address or purpose. A WebSocket URL selects the route but does not hide HTTP/WebSocket setup. Bounded HELO framing and malformed-input handling are still required and belong to Q2/Q10; removing autodetection does not remove those requirements.
### Q9 — Configurable size noise — RESOLVED
**Add configurable random-length filling with random bytes to outgoing protocol records.** Apply padding inside encryption and authentication, including its delimiting/length information, so the receiver can recover the object unambiguously and an attacker cannot expose the true size by reading a padding-length field. The MAC/authentication covers the entire padded body. No unauthenticated filler is appended outside the protected record.
Expose an application configuration controlling the amount of padding, including a disabled option. Section 6 now supplies configurable starting values selected under the user's delegation; Q2 records their exact wire encoding. Keep total padded sizes within configured frame/allocation limits and the UDP datagram budget. Consider handshake records as well as application records when specifying coverage.
“Packet” here means a protocol record/message we construct, not an individual TCP segment: the operating system may split or combine writes. Padding is useful over WebSocket too: it adds uncertainty to visible message sizes even though HTTP/WebSocket presence remains recognizable. It does not hide timing, endpoints or total traffic, and repeated observations may still reveal statistical size patterns. No cover traffic or timing shaping is implied by this decision.
### Q10 — UDP handshake and resource handling — RESOLVED AT DESIGN LEVEL
Use bounded attempts and state. A new attempt always generates fresh ephemeral material and HELO, as already agreed. No automatic application-RPC retry is introduced: repeating an operation after an uncertain result is application policy.
- Lost HELO/server hello: expire the attempt and retry with a new DH pair. The recent-DH guard continues silently ignoring repeated HELOs.
- Lost client ACK or server acceptance: permit bounded retransmission of the exact protected ACK within the same attempt. Retain its authenticated identity and the exact acceptance ciphertext briefly. An identical authenticated ACK may trigger retransmission of that cached acceptance, without rerunning identity hooks, authorization, connected callbacks or application commands. Compare the exact authenticated flight/session association; do not treat arbitrary replayed data as a handshake retry. Normal data replay filtering remains unchanged.
- Do not autonomously flood acceptance messages. Resend only in response to the matching ACK and within the retry/rate budget.
- Bind incomplete UDP attempts to the receiving listener and source address/port. No transparent migration/NAT-rebinding mechanism in the initial version; reconnect with fresh keys. Do not allow unverified packets to replace an established session for that address. The detailed adapter/session-dispatch implementation must preserve this rule.
- Before return-path confirmation, bound outgoing bytes to at most three times bytes received for the attempt; there is only one server-hello response to a fresh HELO. Size/pad the legitimate HELO sufficiently for the supported response within the datagram budget. Successful decryption alone is not return-path validation: a sender knowing the public key can spoof an address. The protected ACK demonstrates receipt of the fresh server flight.
- Apply bounded admission before expensive public-key work. Use a finite worker pool without an unbounded waiting queue, per-source/global rate budgets and pending-state limits. Over-limit unauthenticated traffic is silently dropped. These controls limit resource use, not malicious access by someone able to create fresh valid HELOs.
- Initial UDP mode has one complete protected record per datagram, no new fragmentation/reassembly subsystem. Oversized messages fail locally with a clear size error; use an explicitly configured stream carrier for larger objects. Padding shrinks to the available budget before rejecting a message whose unpadded record still cannot fit. Do not silently downgrade protection.
Defaults and tunable bounds appear in section 6. UDP size guidance is informed by [RFC 8085](https://www.rfc-editor.org/rfc/rfc8085.html), which recommends avoiding fragmentation and congestion-unresponsive traffic. The selected 1200-byte default is our conservative starting budget, not a universal path-MTU guarantee or a claim that pacing alone supplies full congestion control. Retain existing transport controls; sustained Internet UDP throughput requires an appropriate congestion-control strategy and validation.
### Q11 — Closure and truncation — RESOLVED AT DESIGN LEVEL
Provide an authenticated internal CLOSE record carrying a bounded reason code. A locally requested graceful close stops new admission, drains already admitted writes up to a configurable deadline, then emits CLOSE and closes the carrier. If the deadline expires or writing fails, abort and release queued/pending operations with failure. No half-close API is introduced initially.
For TCP, a valid CLOSE at a record boundary signals clean channel termination. EOF without it is an abrupt/truncated connection; a partial record or bad MAC is a failure. None of these imply that pending RPCs succeeded. CLOSE can use a barrier after the queued records rather than bypass them through priority scheduling.
For UDP, CLOSE is best-effort, sent once; loss is possible and does not justify adding a reliable teardown protocol. Its reception means the sender closed, not that all preceding datagrams arrived. Reordering must not convert pending operations into successes. Retain existing optional liveness mechanisms; no mandatory idle heartbeat or idle-session timeout is added.
### Q12 — Integration and rollout — RESOLVED AT DESIGN LEVEL
Implement an internal protected-channel boundary around complete encoded RPC blocks. It receives explicit local priority metadata and exposes authenticated plaintext blocks. Reuse RPC correlation/dispatch and application interfaces; bypass the existing L0Call encryption wrappers for the new flavor. One scheduling owner per connection; no second priority queue that can reorder already stream-encrypted bytes.
Use separate explicit endpoint factories/configuration for the concealed flavor. Existing endpoints, public API behavior and the light TS package remain unchanged. Configuration is passed through a single immutable options object with grouped policies; reuse existing reconnection/health settings where their semantics match. No automatic fallback to the existing protocol.
During KMP prototyping, consume crypto2 0.9.6-SNAPSHOT containing the DH length guard. Do not update the currently released Kiloparsec branch merely as a documentation change. A stable concealed-protocol release must pin a qualified stable crypto2 release rather than a snapshot. Investigate the recorded JS loader failure before claiming Kotlin/JS support; the separate TypeScript client is not being changed.
### Q13 — Kotlin Elligator — IMPLEMENTATION/VALIDATION TASK
Proceed with a Kotlin prototype using existing libsodium and bignum, without adding a runtime dependency. Test mapping and shared-secret compatibility against independent vectors/implementations. Verify mixed-order input behavior through the actual platform bindings. Treat timing suitability as a dataflow and threat-model question, not something resolved by arithmetic tests alone. The [timing assessment](CONCEALED-PROTOCOL-TIMING.md) records the current public-coordinate dataflow, JVM/Linux Native measurements and the proposed 50 ms roundup experiment; variable execution time is not itself evidence of private-key leakage.
Use fixed test vectors and a fixed-seed randomized corpus (at least 10,000 round trips in the extended suite), boundary/exceptional cases and malformed inputs. Test mask/sign/padding choices and low-order rejection. Measure timing, allocations and retained binary size on JVM and Linux Native. Statistical distinguishers detect regressions; they do not prove cryptographic indistinguishability or constant-time execution.
If these investigations show general bignum unsuitable, implement reviewed fixed-size Kotlin field arithmetic and validate it before production. This stays within the no-new-runtime-dependency direction. Any later decision to add a native library or weaken a security property requires a new design decision; do not silently do either.
**2026-10-22 user decision: bounded HELO failure work.** Each shared server identity keeps an independent random throwaway X25519 scalar seed. Only a failed HELO envelope authentication triggers one extra decryption attempt, using a temporary seed copy with 1–8 distinct effective scalar bits flipped. Discard every dummy outcome; preserve the original authentication rejection. Do not modify the real key, send a decoy response, invoke identity hooks, recursively retry or admit the peer. Erase the temporary key copy after use. Keep the attempt within existing crypto admission bounds. `ConcealedOptions.Handshake.decoyOnHelloAuthenticationFailure` defaults to `true` and can disable this local policy. No wire format change, clock-padding policy or random division loop is introduced. The [timing assessment, section 7](CONCEALED-PROTOCOL-TIMING.md#7-requested-server-side-decoy-decryption) records tests and measured cost. This is bounded failure-path work, without a claim that key mutation defeats every timing or microarchitectural attack.
### Q14 — Qualification — IMPLEMENTATION/VALIDATION TASK
Keep the new flavor experimental until the wire specification and V1–V7 checks pass. First qualify JVM and Linux Native; compile other configured KMP targets and label runtime support according to evidence. Do not advertise browser/mobile/Apple runtime qualification based only on compilation. Leave the current protocol available throughout rollout.
Use deterministic wire vectors across implementations, negative handshake/MAC tests, bounded-input fuzzing, real loopback UDP impairment, TCP segmentation/coalescing, concurrent duplex and fairness tests. Extended CI starts with 10,000 deterministic malformed-input cases and a 60-second bounded duplex/priority run; soak durations and seeds are test-harness settings, not wire parameters. Expand these workloads based on findings rather than treating their counts as a security proof.
Before calling this a production concealed protocol, review the full handshake and custom stream-plus-MAC construction independently of the implementation, including the KDF from concealed clientDH, identity/session binding, rejection behavior, resource limits, truncation and forward-secrecy claims. This review is work still to do, not a completed audit. No further user design question is needed to begin the specification/prototype.
## 6. Configurable implementation defaults
These are assistant-selected starting defaults under the user's instruction to settle routine choices. They are not protocol standards or measured optimal values. Keep them in one immutable `ConcealedConnectionOptions` with grouped `handshake`, `limits`, `padding`, `queue`, `udp` and `close` policies (names provisional). Validate ranges at construction and expose application overrides. Do not change existing-flavor defaults.
| Setting | Initial default | Semantics |
| --- | --- | --- |
| Handshake attempt deadline | 15 seconds | Whole attempt, including hooks; override for interactive/external authorization. |
| Maximum connection attempts | 3 | Includes first attempt; finite by default, fresh DH/HELO each time. |
| Delay between fresh attempts | 700 ms, exponential backoff capped at 5 s, ±20% jitter | Align base delay with existing reconnection configuration; cancellation interrupts waits. |
| UDP ACK retry interval | 1 s, doubling to 4 s, ±20% jitter | Only after a valid server hello; at most 3 retransmissions and never past attempt deadline. |
| Incomplete server state lifetime | 15 seconds | Starts at admission, not reset indefinitely by duplicate traffic. |
| Cached acceptance lifetime | 30 seconds | Expire cryptographic retry state independently of established session lifetime. Bound cache entries. |
| Recent-DH guard | 4096 entries | Optional; 0 disables. Ring eviction, no persistence requirement. Operational only. |
| Pending handshakes | 1024 global / 8 per source IP | Per-listener starting budgets; shared-NAT deployments may raise the per-source limit. |
| Acceptance retry cache | One bounded entry per active server connection | Lifetime configurable; total bounded by listener carrier limits. No independent unbounded map. |
| Bootstrap crypto concurrency | 4 per listener | Reject excess before expensive work; configurable for server CPU capacity. |
| New-HELO admission | 100/s global, burst 200; 10/s per source IP, burst 20 | Bounded token buckets; expire inactive source entries and cap source table at 4096 entries. Reject new entries when full rather than allowing unlimited allocation. |
| Maximum UDP datagram payload | 1200 bytes | Includes all protocol/crypto overhead and padding; configurable for known paths, obey platform/protocol absolute bounds. No claim of universal MTU safety. |
| Maximum bootstrap frame on a stream carrier | 16 KiB | Bound first-message parsing before a session exists. UDP uses its stricter datagram limit. |
| Maximum protected stream body | 16 MiB | Includes inner object delimitation and padding, excludes fixed outer length/tag; checked before allocation. |
| Length encoding | Fixed unsigned 32-bit, big-endian | Encrypted 4-byte outer body length; authenticated inner object byte length before object+padding. Avoid unbounded varint parsing. |
| Partial stream-record deadline | 30 seconds | Starts when the first byte of a new record arrives; does not time out an otherwise idle connection. Configurable for slow links/large records. |
| Outgoing queue | 128 items / 32 MiB | Reuse current writer defaults; enforce byte budget including protection/padding cost. |
| Maximum priority overtaking | 2730 admission positions | Reuse OutQueue behavior; all regular entries numbered. |
| UDP replay window | 4096 sequence positions | Existing baseline; local overtaking <= floor(2*window/3). |
| Random padding | Uniform 0–255 bytes per record | Fresh random length and bytes, protected with body. Explicit disabled option; effective maximum capped by carrier/record/queue budget. Handshake minimum padding for response budget is separate. |
| Graceful-close drain deadline | 2 seconds | Abort if drain/close cannot finish; no false success for unanswered RPCs. |
| Heartbeat | Disabled unless application enables existing health policy | Do not add idle polling as part of concealment. |
Padding layout for stream records becomes `encrypted(outerBodyLength | innerObjectLength | objectBytes | padding | tag)`, with the tag covering the implicit sequence, exact outer length and entire body. Verify before interpreting inner length or decoding the object. Outer length is fixed-width but still untrusted until authentication; cap the read and deadline. The same protected inner object-length/body convention can be used in independent UDP records, alongside their encrypted sequence number. Specify exact control/message types in Q2.
Local receive limits and any peer-advertised limits must be reconciled in the authenticated handshake (use the smaller applicable bound); never allocate from a peer claim beyond the local cap. The bootstrap starts with bounded endpoint-configured limits before negotiation. Do not silently change wire encodings or crypto algorithms through ordinary tuning parameters.
The 16-byte MAC tag, 24-byte UDP IV, key separation, identity verification, DH-error rejection, authenticated-before-dispatch rule and counter non-wrap are fixed security properties, not options to disable. For the first wire version, keep the replay window at 4096 and validate configurable overtaking against it; changing the window requires explicit authenticated negotiation/specification support. Specify cipher usage ceilings/rekeying in Q2; fail closed before limits and reconnect with fresh keys rather than reset counters under an old key.
## 7. Proposed validation sequence
1. **V1 — Elligator compatibility:** reference vectors; encode/decode; sign/top-bit choices; exceptional inputs; randomized subgroup coverage; agreement between Kotlin/libsodium and an independent implementation.
2. **V2 — Bootstrap/handshake:** mutation of every authenticated field, replayed HELO/ACK, substituted DH, wrong server identity, rejected client authorization, transcript/role confusion, all-zero DH rejection, safe failure behavior.
3. **V3 — TCP:** arbitrary segmentation/coalescing and partial cipher-mask blocks; authenticated length mutation; truncation; MAC changes; deadlines; size limits; no object deserialization/dispatch before verification; cancellation mid-write.
4. **V4 — UDP:** duplicate, stale, reordered, lost and replayed records; independent decryption; bounded handshake retransmission/cache behavior; nonce/sequence exhaustion handling.
5. **V5 — RPC/lifecycle:** duplex calls, responses, errors, pushes, initialization RPCs, authorization rejection, connection notification and close semantics.
6. **V6 — Scheduling/performance:** sustained priorities with low-priority progress, large-message latency, bounded queues, concurrent producers, allocation/cipher/MAC cost and actual artifact-size delta.
7. **V7 — Platform/security review:** JVM and Linux Native first, then the explicitly supported KMP targets; reference implementation outside production dependencies; randomness distinguishers as regression tests, not proofs; review the custom stream-plus-MAC composition before release.
## 8. Sources consulted
- [Elligator paper](https://elligator.cr.yp.to/elligator-20130828.pdf)
- [Elligator key-exchange construction](https://elligator.org/key-exchange)
- [Elligator direct/inverse maps](https://elligator.org/map)
- [Monocypher Elligator manual](https://monocypher.org/manual/elligator) — reference, not chosen runtime dependency.
- [Monocypher security disclosures](https://monocypher.org/quality-assurance/disclosures) — use a reference version including applicable timing fixes.
- [Libsodium point arithmetic](https://libsodium.gitbook.io/doc/advanced/point-arithmetic)
- [Libsodium sealed boxes](https://doc.libsodium.org/public-key_cryptography/sealed_boxes)
- [Libsodium key exchange](https://doc.libsodium.org/key_exchange)
- [Libsodium stream ciphers](https://doc.libsodium.org/advanced/stream_ciphers)
- [Libsodium secretstream](https://doc.libsodium.org/secret-key_cryptography/secretstream) — investigated alternative; still requires ciphertext chunk boundaries, not selected for the current TCP plan.
- [RFC 7748](https://www.rfc-editor.org/rfc/rfc7748.html) — X25519 encoding and DH result checking.
- [RFC 8017](https://www.rfc-editor.org/rfc/rfc8017.html) — RSA ciphertext range.
- [Ionspin KMP bignum](https://github.com/ionspin/kotlin-multiplatform-bignum)
- [RFC 8085: UDP usage guidelines](https://www.rfc-editor.org/rfc/rfc8085.html) — datagram sizing, reliability and congestion considerations.
An experimental source implementation and wire draft now exist; see the implementation record for completed validation and remaining qualification. The design below preserves the decisions and rationale, not a production-security claim. Q1/Q3–Q12 are resolved at design level. Q2 is the wire-specification task; Q13/Q14 and V1–V7 are implementation/validation work, not completed security claims. The concrete specification, prototype and expanded validation now exist; the [qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md) is the current continuation point. Retain the stable identifiers.
+147
View File
@@ -0,0 +1,147 @@
# Experimental concealed protocol: implementation and continuation
The design is recorded in [CONCEALED-PROTOCOL-DESIGN.md](CONCEALED-PROTOCOL-DESIGN.md); exact byte layouts and cryptographic inputs are in [CONCEALED-PROTOCOL-WIRE.md](CONCEALED-PROTOCOL-WIRE.md). This implementation is experimental. The custom handshake/stream composition has not received an external cryptographic audit. Elligator uses variable-time bignum arithmetic on public coordinates in the current call graph; the [timing assessment](CONCEALED-PROTOCOL-TIMING.md) narrows the original warning and records measured variation without claiming a demonstrated private-key leak.
## Implemented components
- Common Kotlin Elligator 2 mapping and full-curve key generation, using the existing bignum/libsodium dependencies. Includes published reference vectors, subgroup and representation tests.
- Four-flight handshake: concealed HELO, protected signed server reply, session-encrypted client identity proof, session-encrypted acceptance. Exact transmitted representatives, both DH keys, options and identities are bound through the transcript. A fresh random 24-byte IV protects each bootstrap/finished envelope.
- Independently decryptable UDP records with random 24-byte IVs, encrypted sequence counters, authenticated padding and the existing replay bitmap.
- Continuous TCP ChaCha20 stream with preserved partial-block state, hidden bounded lengths and keyed BLAKE2b-128 record authentication. The existing binding does not expose portable XChaCha20 stream operations, so a derived 8-byte nonce is used with the original ChaCha20 64-bit block-counter API. This does not change UDP's 24-byte IVs.
- Complete-block integration with the existing Transport RPC engine. Ciphertext never passes through the legacy L0Call wrapper. One priority scheduler owns admission and sending. TCP authenticates/encrypts after queue selection; UDP protects before admission. Both reuse OutQueue's bounded-overtaking algorithm.
- Deferred readiness, application credential policy through the existing non-suspending KiloInterface hooks, client/server scope identities and tokens, duplex RPC, pushes, exception mapping, protected closure, finite client establishment retries and no automatic retry of application operations.
- Raw TCP, UDP and binary WebSocket carriers, plus bounded listener admission on sockets. The old TCP/UDP envelopes and keepalive bytes are absent from the new socket adapters.
The exact footprint and completed validation are recorded below. No publication or deployment is implied by these source changes.
## Using the experimental API
Initialize crypto as usual. The server's encryption key and identity key are separate and supplied by the application. Share one `ConcealedServerIdentity` between accepted connections to share its recent-DH guard and cryptographic concurrency budget. Key provisioning and rotation remain application policy.
```kotlin
@OptIn(ExperimentalConcealedProtocol::class)
suspend fun serve(
api: KiloInterface<Unit>,
encryptionKey: DecryptingSecretKey,
signingKey: SigningKey,
) = coroutineScope {
val identity = ConcealedServerIdentity(encryptionKey, signingKey)
val listener = ConcealedTcpListener(port = 7443)
try {
listener.carriers.collect { carrier ->
launch {
try {
ConcealedConnection.server(api, carrier, Unit, identity).run()
} catch (_: CancellationException) {
// Parent shutdown.
} catch (_: Exception) {
// Local logging/metrics policy; do not reply in plaintext.
} finally { carrier.close() }
}
}
} finally { listener.close() }
}
```
Client establishment retries use a new carrier and fresh ephemeral material on each attempt:
```kotlin
@OptIn(ExperimentalConcealedProtocol::class)
suspend fun connectExample(
api: KiloInterface<Unit>,
serverEncryption: EncryptingPublicKey,
trustedServerIdentity: VerifyingPublicKey,
clientIdentity: SigningKey,
) = coroutineScope {
val options = ConcealedOptions(
padding = ConcealedOptions.Padding(maxBytes = 255),
handshake = ConcealedOptions.Handshake(maxAttempts = 3),
)
val client = ConcealedClient(api, Unit, serverEncryption,
trustedServerIdentity, clientIdentity, options) {
connectConcealedTcpCarrier(NetworkAddress("server.example", 7443))
}
val runner = launch { client.run() }
try {
client.awaitReady()
// client.call(command, args), client.push(command, args)
} finally {
client.close()
runner.cancelAndJoin()
}
}
```
`ConcealedConnection.client(...)` is the lower-level single-attempt form. A `ConcealedConnection`/`ConcealedClient` instance is run only once. `ConcealedClient` retries establishment failures within its finite policy, stops on authenticated signature/policy failures, and does not reconnect/reissue an RPC after an established connection fails. The caller decides how business operations recover from an uncertain result.
Raw TCP/UDP APIs are available on the source sets configured for Ktor sockets (including JVM, Android, Linux and Apple targets). Common JS/Wasm code can use supported WebSocket or custom carriers; compilation alone does not establish browser runtime qualification.
For UDP use `ConcealedUdpListener` / `connectConcealedUdpCarrier`. One protected message fits one datagram. The default complete-datagram budget is 1200 bytes; oversize objects fail locally. This implementation does not introduce fragmentation or reliable delivery for ordinary RPCs.
For WebSocket use `connectConcealedWebSocketCarrier(url)` or a caller-owned Ktor WebSocket session's `asConcealedCarrier()`. The server selects the explicit route and installs Ktor WebSockets. Protocol bytes use binary frames; frame boundaries are just stream chunk boundaries. Caller-owned HTTP clients remain caller-owned. Padding obscures message-size relationships, while the HTTP/WebSocket handshake remains visible.
Existing `onConnected` hooks are ordinary, non-suspending functions. They may reject credentials or launch work that awaits RPC readiness. They must not block the event loop waiting for an RPC on the same not-yet-ready connection. No new suspending authorization callback or special pre-establishment RPC capability is introduced.
## Configuration and fixed properties
`ConcealedOptions` groups handshake, framing/padding, queue, UDP and closure limits. Socket listeners expose their own carrier-count, source-table and new-connection rate budgets; `ConcealedServerIdentity` exposes recent-DH capacity and crypto concurrency. Keep them associated with a listener, not rebuilt per packet.
Receive body size, datagram budget and padding ceilings are advertised inside authentication and reconciled with local bounds. Queue limits count admitted encoded data; stream protection adds bounded padding and authentication overhead when the selected record is written. At most one record is encrypted/written outside the queue at a time. A large record already writing is not preemptible.
Server HELO authentication failures now perform one discarded decryption with a mutated copy of a separate random throwaway key. This is enabled by default through `Handshake.decoyOnHelloAuthenticationFailure`; it remains inside the existing crypto admission limit, cannot authenticate a peer or send a response, and does no extra work for successful HELOs. It roughly doubled failed-HELO CPU time in the measured JVM run; see the [timing assessment](CONCEALED-PROTOCOL-TIMING.md#7-requested-server-side-decoy-decryption) for exact behavior, tests and limits.
The 16-byte authentication tag, identity checks, DH rejection, role separation, verified-before-decoding rule and fail-before-counter-wrap behavior are not disable switches. Per-direction limits additionally cap records at 2^32 and TCP encrypted bytes at 1 TiB; usage exhaustion requires a new session, not counter reset under an old key.
Low-level record state is externally serialized by the connection. Do not share a cipher/handshake instance between connections or restart it with reused keys/initialization material. Close/cancel the old run before abandoning its carrier.
## Build and dependency details
- The prototype consumes `crypto2:0.9.6-SNAPSHOT`, which contains the peer-DH length guard. This snapshot was compiled with Kotlin 2.4, so the project and its included interoperability fixtures now use Kotlin 2.4.0 as well.
- The existing `multiplatform-crypto-libsodium-bindings:0.9.6` backend is resolved from the YoungBlood Maven repository; keep that repository alongside SergeychWorks, Maven Central and Universa as shown in the main README. A standalone private-Maven-only consumer exposed the missing repository in its initial fixture; adding the existing backend repository resolved it.
- Direct use of the existing libsodium backend is explicit in the build. No additional crypto runtime library such as Monocypher was added.
- The existing libsodium binding eagerly captures `_sodium_init`. In JS runtime 0.7.16 that export appears only after readiness, so the build pins transitive `libsodium-sumo` to compatible 0.7.15 for Kotlin JS/Wasm. This is isolated from the separate lightweight TypeScript package. A proper initialization fix exists in the separate bindings repository and can be qualified later.
- A stable release must use a qualified stable crypto2 dependency and complete the qualification tasks. The first concealed distribution is 0.9.0-SNAPSHOT; see the [qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md) for publication status and exact checks.
## Remaining qualification
Complete the independent protocol review, review the documented Elligator timing/dataflow assessment across supported targets, extend malformed-input fuzzing and sustained-load tests, and qualify platform runtimes according to actual evidence. Deterministic tests and randomness checks are valuable regression evidence, not proofs of cryptographic security.
The first implementation has no automatic key rotation, shared-port flavor detection, UDP fragmentation, address migration, mandatory heartbeat, cover traffic or timing shaping. These decisions are recorded in the design and are not silently supplied by the encryption layer.
## Verified run: 2026-10-22
- Full JVM suite: **109 passed**, including existing protocol regressions, raw TCP/UDP loopback, and real Netty WebSocket tests.
- Full Kotlin/JS Node suite: **82 passed**, with the compatible libsodium runtime resolution.
- Full Linux x64 Native suite: **85 passed**.
- Compilation passed for Android release, Wasm, Linux arm64, Windows x64, macOS arm64/x64, and iOS arm64/x64/simulator arm64. These compile checks do not qualify device runtimes or final Apple application linking.
- TypeScript unit suite: **38 passed**, one source-server interoperability test intentionally skipped by its ordinary unit command; the separate live source-JVM interoperability command passed pinned identity, duplex RPC, pushes, errors, parallel calls and reconnection.
- Tests cover independent Elligator/KDF/MAC vectors, all eight torsion components, alias encodings, signatures/transcripts, bad DH inputs, duplicate/stale records, lost UDP handshake flights, unauthenticated noise, negotiated limits, deadline trickling, deferred initialization, cancellation, finite retries, uncertain-operation behavior and explicit key cleanup with rejection of destroyed record state.
- Native testing exposed an existing empty-response hazard: the generic `utools.unpack` helper maps empty bytes to generic null. The transport's encrypted-response and complete-block network boundaries now use strict BiPack decoding, avoiding null-backed unsigned arrays. The legacy forged-ACK regression passes on all three tested runtimes.
- The unshrunk JVM JAR is 828,075 bytes versus the published 0.8.0 baseline's 537,671 bytes: **+290,404 bytes (about 284 KiB)**. This includes the prototype, raw carriers, compiler/dependency metadata changes and coroutine classes. It is not an APK, native executable or tree-shaken browser size measurement. No additional crypto runtime library was added.
Validation command (using JDK 21 and configured SDK/toolchains):
```sh
./gradlew '-Dorg.gradle.jvmargs=-Xmx2g -XX:MaxMetaspaceSize=1g' \
jvmTest jsNodeTest linuxX64Test compileKotlinWasmJs compileReleaseKotlinAndroid \
compileKotlinLinuxArm64 compileKotlinMingwX64 compileKotlinMacosArm64 \
compileKotlinMacosX64 compileKotlinIosArm64 compileKotlinIosX64 \
compileKotlinIosSimulatorArm64 jvmJar
```
The combined command completed successfully after the final lifecycle cleanup. A legacy WebSocket test was changed to await the actual disconnected state with a real-time deadline instead of using a virtual delay to wait for socket IO. New WebSocket test fixtures own their server coroutine scopes, observe peer closure, and have bounded coroutine/JUnit deadlines. An earlier failed run exposed that fixture cleanup issue; it was corrected before the successful run. Existing JS/Wasm lockfile policy may require `kotlinUpgradeYarnLock` when changing the Kotlin toolchain; the libsodium runtime is explicitly pinned by the build.
Remaining work before a production release: independent full-protocol review, target-specific Elligator timing/dataflow qualification and any hardening justified by those findings, extended fuzz/soak workloads and actual browser/mobile/Apple runtime tests. This paragraph records the initial implementation checkpoint; current source/distribution status is in the [snapshot qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md#7-distribution-verification).
## Subsequent timing and failure-path validation: 2026-10-22
The [timing report](CONCEALED-PROTOCOL-TIMING.md) records opt-in JVM/Linux Native sampling, actual handshake-stage measurements, and the exact 50 ms grid plus whole-millisecond jitter experiment. The original timing warning was narrowed: the current bignum call graph operates on public coordinates, not the server's private scalar.
After adding the requested bounded decoy behavior, focused `HelloFailureNoiseTest` and `ConcealedConnectionTest` runs passed **11 tests each on JVM, Kotlin/JS and Linux Native**. Android release and Wasm compilation passed. A separate actual-path JVM timing probe passed with 64 samples per class. These are subsequent focused checks, separate from the earlier full-suite counts and JAR size recorded above. No artificial delay or random division loop was added.
The subsequent 0.9.0-SNAPSHOT release qualification passed 127 JVM tests (including extended campaigns), 90 Kotlin/JS tests and 94 Linux Native tests. [The qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md) includes independent composition vectors, real socket payload captures, optimized Native performance, allocation data and publication verification.
Published 0.9.0-SNAPSHOT to Maven Local and private Maven; all 13 publications / 58 downloaded files passed SHA-256 verification. A standalone application consuming only private-Maven dependencies passed actual TCP duplex RPC. API documentation was deployed and verified. See [distribution verification](CONCEALED-PROTOCOL-QUALIFICATION.md#7-distribution-verification) for precise evidence and repository requirements.
+110
View File
@@ -0,0 +1,110 @@
# Concealed protocol: 0.9.0-SNAPSHOT qualification
2026-10-22. This is the continuation point for the first snapshot release. It records completed checks and their limits, separate from the historical [design](CONCEALED-PROTOCOL-DESIGN.md), [wire specification](CONCEALED-PROTOCOL-WIRE.md), [API guide](CONCEALED-PROTOCOL-IMPLEMENTATION.md) and earlier [timing assessment](CONCEALED-PROTOCOL-TIMING.md).
## 1. Release decision
Publish **0.9.0-SNAPSHOT**, with the concealed flavor opt-in on separate endpoints. Existing protocol 2 remains available; the lightweight TypeScript client stays at 0.8.0. The project now requires Kotlin 2.4.0 and consumes crypto2 0.9.6-SNAPSHOT. This is the first distributed concealed implementation, not a frozen stable protocol or a universal runtime qualification.
No new crypto runtime dependency was added. The explicit libsodium binding is already the crypto2 backend; the independent Python reference is a development tool only.
## 2. Independent composition verification
The [reference and fixture guide](CONCEALED-PROTOCOL-VECTORS.md) records 52 deterministic fixture fields and their SHA-256. Python independently implements the protocol labels, lengths, hashes, ordered KX derivation, signatures, F1–F4 envelopes, stream cursor and UDP encoding. Its continuous ChaCha20 implementation is pure Python; trusted curve/box/signature primitives use system libsodium.
Three common tests passed on JVM, Kotlin/JS and Linux Native. They compare exact production component output with the reference, exercise partial cipher-mask boundaries at offsets 27/121/148, decode reordered UDP, and pass independent F1 through the real server state machine. Fixed F2–F4 comparisons exercise components; complete client/server state machines are additionally covered by randomized integration tests. No production entropy override was added.
Source review checked exact HELO binding, the signed server core, role-assigned session keys, client signature/token binding, h3/traffic-key association and authentication-before-replay-state updates. It found no new composition defect. This is source review and executable cross-checking, not an external audit of the primitives or protocol.
## 3. Adversarial and sustained-load tests
`ExtendedConcealedQualificationTest` is enabled with `KILOPARSEC_EXTENDED_TESTS=1` and passed all four campaigns:
1. **10,000 malformed inputs**, seed `0x237d404`: 2,500 random unauthenticated datagrams, 2,500 authenticated inconsistent inner lengths, 2,500 modified stream records and 2,500 truncated stream records. No unauthenticated object was delivered; invalid datagrams did not consume replay state; stream failures were terminal. Inputs and allocation sizes were bounded.
2. **12,800 UDP records**, seed `0x51a7`: 11,636 delivered, 1,164 intentionally dropped, 2,327 duplicate drops, plus tag forgery before each delivered packet and a stale replay after window eviction. Shuffling retained at most 128 packets at a time.
3. **Admission campaign:** 10,000 attempts respected a global burst of 200; refill admitted 100 in the next simulated second; a 64-entry source table rejected rotating new sources until idle expiry. This targets the actual admission implementation directly, not 10,000 expensive full HELO decryption attempts.
4. **60-second duplex priority run:** eight RPC workers and a low-priority push worker over 16-capacity channels. Low-priority calls and pushes made progress in every 10-second interval. Explicit item and byte queue saturation rejected excess admission, and shutdown completed within its deadline. Results are workload-specific, not a general throughput guarantee.
## 4. Real socket payload capture
`ConcealedTrafficCaptureTest` passed eight fresh TCP and eight fresh UDP sessions through forwarding proxies, with five duplex echo calls per session. It captured actual loopback socket payloads without privileged packet access. All 16 first-flight representative prefixes differed. Neither the repeated synthetic RPC plaintext marker nor either identity public key nor the static server encryption public key appeared in captured directional payload streams.
Local artifacts are under `build/concealed-qualification/traffic/`: binary chunks, `capture.csv` with monotonic timestamps/sizes/directions, and `summary.txt`. TCP boundaries here are proxy read chunks; UDP boundaries are complete datagrams. This is not an IP-level packet capture or a traffic-classifier evaluation. Endpoints, timing, volume and statistical length/flight patterns remain observable. WebSocket upgrade visibility remains documented and its real Netty regression tests passed, but it was not included in this payload-capture campaign.
The first capture fixture incorrectly expected a locally cancelled socket read to leave an empty terminal diagnostic after deliberate shutdown. The assertion now checks connection health before shutdown, and all subsequent captures and the full suite passed. No protocol code was changed to obtain that result.
## 5. Platform results
The final combined command passed:
| Check | Result |
| --- | --- |
| Full JVM suite, extended tests enabled | 127 passed |
| Full Kotlin/JS Node suite | 90 passed |
| Full Linux x64 Native suite | 94 passed |
| Android release and Wasm | Compilation passed |
| Linux arm64, Windows x64 | Compilation passed |
| macOS arm64/x64, iOS arm64/x64/simulator arm64 | Compilation passed |
| Dokka HTML | Generated successfully |
| TypeScript unit suite | 38 passed, one ordinary-command interop skip |
| Live TypeScript/JVM interoperability | Passed pinned identity, duplex RPC, pushes, errors, parallel calls and reconnect |
Compilation is not browser/device runtime qualification or final Apple application linking. Existing unrelated compiler/deprecation and Dokka link warnings remain non-fatal.
```sh
KILOPARSEC_EXTENDED_TESTS=1 ./gradlew \
'-Dorg.gradle.jvmargs=-Xmx2g -XX:MaxMetaspaceSize=1g' \
jvmTest jsNodeTest linuxX64Test compileKotlinWasmJs compileReleaseKotlinAndroid \
compileKotlinLinuxArm64 compileKotlinMingwX64 compileKotlinMacosArm64 \
compileKotlinMacosX64 compileKotlinIosArm64 compileKotlinIosX64 \
compileKotlinIosSimulatorArm64 dokkaGeneratePublicationHtml --console=plain
```
## 6. Optimized Native measurements
The optimized Linux x64 test binary was linked using `NativeBuildType.RELEASE`, through an explicit optional Gradle test run. The timing report identifies it as release and is stored separately from the debug run (`build/concealed-timing/linuxX64-release.csv`). Sampling used 256 measurements per class after 64 warmup rounds on the same Intel Core Ultra 9 host, with no simultaneous load test or other benchmark from this task. CPU frequency scaling and unrelated workstation activity were not disabled.
| Operation | Native release median | p99 | Maximum |
| --- | ---: | ---: | ---: |
| Decode random public representative | 6.741 ms | 8.765 ms | 8.986 ms |
| Encode representable coordinate | 12.736 ms | 14.772 ms | 15.187 ms |
| Reject unrepresentable coordinate | 5.954 ms | 7.382 ms | 8.052 ms |
| Generate fresh Elligator key pair | 14.603 ms | 57.171 ms | 59.791 ms |
The corresponding earlier debug generation median/p99 were 154.524/537.195 ms. The optimized result is about 10.6 times faster at the median; neither run sets a universal deadline. Inverse-map rejection counts still produce expected generation tails. This is a Linux x64 measurement, not an Apple/Android performance claim.
```sh
KILOPARSEC_TIMING_PROBE=1 KILOPARSEC_TIMING_FLAVOR=release \
KILOPARSEC_TIMING_SAMPLES=256 KILOPARSEC_TIMING_WARMUP=64 \
./gradlew '-Dorg.gradle.jvmargs=-Xmx2g -XX:MaxMetaspaceSize=1g' \
-PconcealedNativeBenchmark=true linuxX64ConcealedBenchmarkTest \
--tests '*ElligatorTimingNativeTest*' --console=plain
```
An additional HotSpot per-thread allocation probe measured 128 real generation/destroy operations after 32 warmups: **17,621,495 mean bytes**, **11,529,472 median**, **34,325,888 p95**, **74,474,504 maximum**. These are temporary managed allocations, including bignum work and varying rejection counts. They exclude other threads and native libsodium allocations, and do not measure retained heap or Native allocation counts. Profiling/optimization is worthwhile for high connection rates; no fixed-field arithmetic rewrite was introduced in this release. Raw output is `build/concealed-timing/allocations-jvm.csv`.
Enable this branch of the existing JVM timing probe with `KILOPARSEC_TIMING_PROBE=1 KILOPARSEC_ALLOCATION_PROBE=1 KILOPARSEC_TIMING_SAMPLES=128 KILOPARSEC_TIMING_WARMUP=32`, filtering `jvmTest --tests '*ElligatorTimingJvmTest*'`. It does not overwrite the timing sample CSV.
The unshrunk candidate JVM JAR is **834,461 bytes**, compared with the published 0.8.0 baseline's 537,671 bytes: **+296,790 bytes (about 290 KiB)**. This includes compiler/metadata/coroutine classes and carriers, not just crypto code. It is not a native executable/APK/browser bundle measurement. Test/reference tools are absent from runtime artifacts.
## 7. Distribution verification
Published all **13 Maven publications** to Maven Local and the private SergeychWorks repository. Every one of **58 files** (JAR/AAR/KLIB, POM, Gradle module metadata, sources, tooling JSON and Apple metadata bridge artifacts) was downloaded from private Maven and matched against the local artifact by SHA-256. The initial timestamped build is `0.9.0-20261021.223610-1`; the normal dependency coordinate is `net.sergeych:kiloparsec:0.9.0-SNAPSHOT`.
Canonical JVM JAR: 834,461 bytes, SHA-256:
```text
dfa3e2123d614eb432b8b949e2d4eac35d2b6a51b904b61f1fcd1e8e66296575
```
A standalone Kotlin/JVM application resolved **only published dependencies**, with no composite/source substitution and no Maven Local repository. It authenticated both peers, completed bidirectional RPC over actual loopback TCP, and loaded the JAR from Gradle's downloaded private-Maven cache. That JAR's SHA-256 matches the verified publication. The first consumer fixture omitted the existing YoungBlood backend repository; adding it resolved the dependency and the README example was corrected. This was a consumer repository configuration gap, not a missing published Kiloparsec artifact.
The API site was deployed using `bin/pubdocs` with strict existing SSH trust. Live index, concealed connection and handshake-options HTML were downloaded and matched to the generated pages by SHA-256. [Published API documentation](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.concealed/index.html).
Local receipts: `build/concealed-qualification/publication.json`, `docs-publication.json`, `final-functional-results/`, and the standalone consumer/build logs under `/tmp`. Source publication includes this report and the deterministic fixtures; raw test captures and operational receipts remain ignored local artifacts. The source release is committed/pushed separately from Maven artifact creation; Git revision records the exact implementation and version declaration.
## 8. Remaining scope
The current evidence supports a controlled SNAPSHOT trial. Remaining work includes real browser/mobile/Apple runtimes, Internet-path deployment behavior and prolonged application-specific load, broader state-machine cross-implementation coverage, and independent review where required for production assurance. Timing variation in public-coordinate mapping is not a demonstrated server-private-key leak. The decoy decryption is supplementary bounded work; effectiveness against a particular Intel microarchitectural attack was not measured here.
The extended suites are opt-in so ordinary tests remain quick. Raw captures, timing samples and publication receipts are ignored build artifacts; this document and the deterministic fixtures preserve reproducible evidence in source control. No real application credentials or customer data were used.
+120
View File
@@ -0,0 +1,120 @@
# Concealed protocol timing assessment
Recorded 2026-10-22. This evaluates the current Elligator call graph, measures its implementation, and tests the proposed wall-clock roundup policy. It is not a demonstration of a private-key recovery attack or a proof that no side channel exists.
## 1. Correcting the original concern
The implementation is variable-time, but the original warning did not adequately distinguish public-input variation from secret-dependent leakage.
- The server calls `decode()` on the representative received in HELO. All bignum inputs in this path are public wire data. The long-lived server scalar enters libsodium's DH operation after that mapping; it does not enter the bignum implementation.
- The client uses libsodium to generate an ephemeral point, then bignum to convert/map its public coordinates. The successful Montgomery coordinate is recoverable from the transmitted representative. The low three torsion-selection bits are cleared by ordinary X25519 and do not contribute to its secret scalar.
- The client retries with independent random candidates when the inverse mapping fails. Discarded candidates are not reused. Elligator's [key-exchange reference](https://elligator.org/key-exchange) explicitly permits this variable retry count for properly random ephemeral keys.
- Client preparation precedes the first HELO. Measuring it remotely requires an independent connection-start reference. Server mapping affects the response path and is directly measurable in principle, but its inputs are public.
Consequently, these measurements must not be described as evidence that our server's private key leaks through bignum arithmetic. Current dataflow does not establish such a leak. This conclusion is based primarily on which values the arithmetic consumes, not on a low statistical score. Future changes that put secret scalars or secret-dependent hidden coordinates into this arithmetic require a new assessment.
## 2. Measurement method and scope
Host: Intel Core Ultra 9 275HX, 24 logical processors, Linux 7.1.8-legion-audio x86_64. JVM: OpenJDK 21.0.12. Kotlin 2.4.0. The original Native measurements below use the Linux x64 **debug test binary**. Subsequent optimized release results and JVM allocation profiling are recorded in the [snapshot qualification report](CONCEALED-PROTOCOL-QUALIFICATION.md#6-optimized-native-measurements).
The Elligator probe uses 64 warm-up rounds and 256 measured samples per class. Class order is shuffled each round with a fixed seed. The eight torsion cases share the same clamped scalar within each round; only the three low bits differ. Received representatives include random public inputs, a zero boundary case, and paired encodings differing only in ignored high bits. Inverse-map cases compare reference inputs that are representable or rejected. Generation uses fresh synthetic randomness; a separate equivalent loop records rejection counts. No key bytes are logged.
The JVM handshake probe performs all four actual cryptographic flights with fresh ephemeral keys and fixed synthetic identities, including the prefix parsing used by the connection. It excludes socket IO, application hooks, admission/queue waits and artificial delays. Its first sample is **not** a process-cold benchmark: earlier tests can have initialized classes. Padding is disabled to isolate cryptographic work.
Monotonic clocks measure elapsed time. Mean, median, p95, p99, maximum and exploratory Welch t statistics are retained with raw samples. Measurements are a single host run, without CPU isolation, a controlled thermal/frequency profile or adversarial co-resident load. GC/JIT, scheduler activity, autocorrelation and multiple comparisons prevent interpreting these numbers as a constant-time certificate.
## 3. Measured results
All times below are milliseconds; 256 measurements per listed group.
| Runtime | Operation | Median | p99 | Maximum |
| --- | --- | ---: | ---: | ---: |
| JVM | Decode random public representative | 2.790 | 4.197 | 5.872 |
| JVM | Encode representable public coordinate | 5.262 | 8.030 | 10.574 |
| JVM | Reject unrepresentable public coordinate | 2.503 | 3.808 | 5.925 |
| JVM | Generate fresh Elligator key pair | 7.603 | 27.407 | 34.354 |
| Native debug | Decode random public representative | 77.316 | 79.015 | 89.529 |
| Native debug | Encode representable public coordinate | 147.269 | 149.589 | 181.251 |
| Native debug | Reject unrepresentable public coordinate | 70.917 | 72.753 | 107.829 |
| Native debug | Generate fresh Elligator key pair | 154.524 | 537.195 | 766.529 |
| JVM | Client HELO preparation | 5.138 | 22.171 | 29.364 |
| JVM | Server prefix authentication | 2.494 | 4.775 | 5.087 |
| JVM | Server HELO response construction | 2.680 | 4.846 | 5.335 |
| JVM | All four crypto flights, without IO | 11.063 | 27.966 | 35.193 |
The eight matched torsion classes had median special-public-key times of **0.335–0.342 ms on JVM** and **6.105–6.198 ms on Native debug**. Exploratory maximum absolute Welch t against torsion class zero was 0.99 and 2.02 respectively. Those small scores are not evidence of constant-time execution; the accepted/rejected public-input encoding branches are deliberately distinguishable (t approximately 57 on JVM and 371 on Native).
The separate instrumented generation loop observed **1–12 candidates on JVM** and **1–13 on Native debug** in 256 runs each. This explains expected rejection-related tails. Its largest observed Native debug generation time was 1.075 seconds; this is a separate sample set from the actual-generator row above.
The revised handshake probe's first-in-probe sample took **198.175 ms**. It was not a controlled process-cold experiment, and startup/JIT/initialization effects were not separately attributed. The remaining warm samples are shown in the table. Neither 50 ms nor the observed warm maximum is a guaranteed execution bound.
Server prefix parsing and subsequent complete-HELO construction currently perform separate public Elligator mappings. Their costs therefore both belong in the actual response path. These measurements do not include the newly added failed-authentication decoy; that cost is measured separately in section 7.
## 4. Exact roundup-plus-jitter experiment
The tested policy is the user's actual proposal, rather than simply adding noise to the current time:
```text
release = ceil(completion / 50 ms) * 50 ms + J
J is uniformly selected from {0, 1, ..., 10} whole milliseconds
```
A deterministic test enumerates all 50,000 microsecond start phases and all eleven jitter choices. For 2 ms versus 3 ms computations, the mean target release still differs by exactly **1 ms**. Subtracting the same mean start time gives the same elapsed-latency difference. Jitter broadens the distributions but does not remove that mean difference under this phase model.
A boundary example is simpler: start at grid phase 47.5 ms. A 2 ms calculation completes at 49.5 ms and releases in 50–60 ms; a 3 ms calculation completes at 50.5 ms and releases in 100–110 ms. These supports are disjoint even after whole-millisecond jitter. Controlling or learning the relevant phase is an additional attacker capability; the example establishes a limitation of the mitigation, not an attack against our handshake.
An operation-start deadline selected **before** calculation, such as `start + 50 ms + J`, gives the same target distribution for both calculations while they fit the budget. Work exceeding the deadline, correlated scheduler delays, and local cache/CPU-contention observations remain outside that guarantee. Rounding the elapsed operation duration into 50 ms buckets also works only while the compared durations stay in the same bucket.
The coroutine experiment uses synthetic 2/3 ms CPU work, 100 samples per class per policy, randomized order and independently randomized grid phase before each sample. It uses a fixed **monotonic** grid, equivalent to a fixed wall-clock grid up to its origin for this experiment, without adjustments of the system wall clock. Jitter is an integer number of milliseconds. Cancellable delays round the remaining wait up to milliseconds and recheck the deadline; scheduler overshoot is measured rather than assumed absent. This is a scheduling experiment, separately labelled from the real Elligator measurements.
| Synthetic CPU work | Unshaped mean | Completion-grid mean | Start-selected deadline mean |
| --- | ---: | ---: | ---: |
| 2 ms | 2.006 ms | 32.465 ms | 54.527 ms |
| 3 ms | 3.006 ms | 34.590 ms | 55.061 ms |
No compute-budget overruns occurred in these synthetic 2/3 ms runs. The 100-sample live comparisons after shaping had absolute Welch t near 1.00 (completion grid) and 1.18 (start deadline), compared with approximately 2574 unshaped. This illustrates how much harder a single small sample set becomes to distinguish. It does **not** invalidate the exact boundary model or establish equal distributions: sampled phases/jitter and scheduler overshoot differ between classes. The exact model, rather than the noisy observed mean gap, establishes the residual grid-boundary effect.
## 5. Interpretation and next decisions
1. **Variable execution time is confirmed.** Different public inputs and accepted/rejected inverse-map branches have different costs. That fact alone is not a private-key disclosure.
2. **No demonstrated secret leak has been found in the current bignum dataflow.** The timing warning should be narrowed accordingly. This does not independently audit libsodium, platform bindings, the custom handshake or co-resident side channels.
3. **Grid roundup is remote timing shaping, not full erasure.** It can make individual timings much less informative. Bucket crossings remain, and independent random extra work, including random bignum divisions, does not supply a general elimination guarantee. Research also demonstrates that concurrent response ordering can expose timing differences despite network noise: [Van Goethem et al., USENIX Security 2020](https://www.usenix.org/conference/usenixsecurity20/presentation/van-goethem).
4. **Do not infer a universal 50 ms budget from this host.** Native debug timings, cold startup, device speed and load differ. Any production deadline policy needs configurable bounds, consistent externally observable success/failure handling, explicit overruns, cancellation and admission limits. Waiting must occur outside the cryptographic semaphore. A global release grid can itself introduce a periodic traffic pattern; this has not been tested as a network fingerprint here.
5. **No clock-padding or random-division policy was added.** Following the subsequent user request, one bounded decoy decryption was added to the HELO authentication-failure path; section 7 records its exact behavior and cost. It is not an authentication upgrade or a general timing-concealment guarantee. An independent protocol review and the remaining fuzz/platform qualification are still required.
## 6. Reproduction and artifacts
Ordinary test runs do not execute the expensive sampling loops. Enable them explicitly:
```sh
KILOPARSEC_TIMING_PROBE=1 KILOPARSEC_TIMING_SAMPLES=256 \
KILOPARSEC_TIMING_WARMUP=64 \
./gradlew '-Dorg.gradle.jvmargs=-Xmx2g -XX:MaxMetaspaceSize=1g' \
jvmTest --tests '*Timing*' linuxX64Test --tests '*ElligatorTiming*' --console=plain
```
`KILOPARSEC_SHAPING_SAMPLES` controls the live synthetic sample count per class/policy (default 100). Fresh randomness in actual generation, CPU state and scheduling mean that repeating the command will not reproduce identical timings. A repeated invocation with unchanged source may be considered up-to-date by Gradle; add `--rerun-tasks` when a fresh sampling run is intended. That also reruns compilation prerequisites.
Local ignored artifacts under `build/concealed-timing/` contain `jvm.csv`, `linuxX64.csv`, `handshake-jvm.csv`, `shaping-samples.csv`, `shaping-summary.txt` and host/run metadata. The tables above preserve the relevant summary in versionable documentation; raw samples remain local build output.
## 7. Requested server-side decoy decryption
Each `ConcealedServerIdentity` now owns a separate random 32-byte throwaway scalar seed. When the real HELO envelope authentication throws `UnauthenticatedFlight`, the wrapper copies the seed, flips 1–8 distinct effective X25519 scalar bits (positions 3 through 253), and performs **one** additional envelope-decryption attempt with that copy. The temporary copy and mutation entropy are cleared afterward. The persistent seed and the real server key are unchanged; concurrent attempts use independent copies.
The dummy outcome is discarded even if it succeeds. It cannot create a response, invoke identity hooks, advance real handshake state or turn a failed authentication into success. Expected dummy authentication/parsing failures preserve the original failure. Cancellation, unexpected runtime failures and fatal errors propagate. Bad lengths, low-order peer inputs and admission rejection do not trigger an envelope-authentication decoy. Client ACK/session-record failures are not part of this HELO-specific mechanism.
Both prefix and full-envelope failure paths run inside the existing server crypto admission limit. There is no recursive retry, added coroutine delay, random division loop or added unauthenticated response. The local server option `ConcealedOptions.Handshake.decoyOnHelloAuthenticationFailure` defaults to `true`; set it to `false` to omit the extra attempt. It is not negotiated and does not change the wire format.
A separate JVM probe measured 64 samples per class after 16 warmups, randomized across the same eight synthetic HELOs. It exercises the actual identity/admission wrapper and real envelope authentication, without network IO:
| Case | Median disabled | Median enabled | Mean added work |
| --- | ---: | ---: | ---: |
| Valid header | 2.488 ms | 2.481 ms | -0.034 ms |
| Bad header authentication | 2.501 ms | 4.981 ms | 2.435 ms |
| Bad body authentication | 2.584 ms | 5.109 ms | 2.646 ms |
This approximately doubles failed-HELO CPU work on the measured JVM, while successful authentication adds no dummy work. X25519 is designed to be constant-time and the repeated Elligator decode consumes the same public representative; changing the dummy scalar therefore does not itself imply broad random timing variation. No Intel microarchitectural attack reproduction was performed, and effectiveness against the user's previously observed attacks has not been established by these tests.
Five new unit tests cover exact invocation bounds, successful dummy-result discard, expected versus unexpected exceptions, effective-bit mutation and erasure, concurrent copy independence, both actual failure stages, and preservation of real keys and a later valid handshake. Existing connection regression tests cover UDP injected noise and readiness behavior. The five new tests plus six connection tests passed on **JVM, Kotlin/JS and Linux Native (11 each)** after the production change; Android release and Wasm compilation also passed. The actual-path JVM failure timing probe passed separately.
The additional opt-in probe is `HelloFailureTimingProbeTest`; output is `build/concealed-timing/helo-failure-jvm.csv`. Run with `KILOPARSEC_TIMING_PROBE=1`; `KILOPARSEC_FAILURE_TIMING_SAMPLES` controls samples per class (default 64). The earlier `*Timing*` JVM filter includes it on subsequent fresh runs.
+42
View File
@@ -0,0 +1,42 @@
# Concealed protocol: independent composition vectors
2026-10-22. These are qualification fixtures for experimental suite 1, not a cryptographic audit or a claim of indistinguishability.
## Independent implementation
`tools/concealed-vectors/reference.py` generates 52 fixture fields without importing Kotlin source, executing Kotlin output, or reading the generated fixture file. All private keys, identity seeds, IVs and plaintexts in these files are deliberately public synthetic test inputs. Never use them for a connection.
The reference uses Python's standard-library `hashlib.blake2b` for hashes, KDFs, MACs, and the explicit `crypto_kx` derivation (`BLAKE2b-512(X25519 || clientPublic || serverPublic)`). Its original ChaCha20 implementation is plain Python with a 64-bit counter and 64-bit nonce. It uses the installed system libsodium through `ctypes` for X25519, XSalsa20-Poly1305 and Ed25519. Thus the protocol composition and stream-cursor implementation are independent, while the underlying trusted curve, box and signature primitives are intentionally shared. There are no added production dependencies or downloaded Python packages.
Reproduce from the repository root:
```sh
python3 tools/concealed-vectors/reference.py
sha256sum tools/concealed-vectors/fixtures.json
```
Expected JSON SHA-256:
```text
7a7acacb100b028a8f78ad933b904cc97afd0f57eac9b1513c9ca6bb01ef0a27
```
Regeneration also writes `IndependentWireFixtures.kt`, consumed on JVM, Kotlin/JS and Native by `IndependentWireVectorsTest`.
## Checks performed by the Kotlin tests
1. Exact F1 bytes, public representative decoding, bootstrap DH, bootstrap root and h1. A real `ServerHandshake` accepts this independently generated HELO and obtains the expected client DH public key. Changing either ignored representative top bit preserves its mathematical point but fails authentication because the exact encoding is bound into the KDF.
2. Exact F2 bytes, server signature, reply KDF and h2.
3. Independent explicit KX derivation agrees with both Kotlin libsodium role APIs; c2s and s2c assignment, token and both finished keys match.
4. Exact client signature, F3, h3 and F4 bytes.
5. Both directional stream bases, salts, MAC keys and UDP keys agree with `makeSession`.
6. Three complete TCP records match the pure-Python ChaCha20/MAC composition byte for byte. Record ends are at offsets 27, 121 and 148, deliberately crossing and retaining partial 64-byte cipher masks. Decoding succeeds at chunk sizes 1, 3, 63, 64, 65 and the entire stream.
7. Independently encrypted UDP records are accepted in order 2, 0, 1; duplicates are discarded and a modified ciphertext is rejected.
Padding is disabled for these byte-exact fixtures. Object bytes are opaque record-engine inputs; they are not examples of valid application RPC serialization. The fixed HELO is for the stream carrier; UDP record fixtures independently exercise datagram protection.
## Scope limits
F2–F4 fixture comparisons exercise the production envelope, signature, KDF and session-material components with fixed inputs. They do not force the production client/server state machines to use deterministic secret keys or IVs. The actual state machines continue to obtain entropy exclusively from libsodium, and separate randomized four-flight and connection integration tests cover their operation. No public or internal production entropy override was added to obtain vectors.
These checks target encoding, key ordering, transcript coverage and cursor errors. They do not prove the custom stream/MAC composition secure, quantify a timing adversary, establish traffic indistinguishability, or replace review of application trust provisioning. The bootstrap private-key compromise boundary remains as designed: recorded bootstrap flights become readable, whereas independently authenticated fresh session DH protects client identity and application traffic, subject to ephemeral-secret erasure.
+309
View File
@@ -0,0 +1,309 @@
# Concealed Kiloparsec: experimental wire specification
Draft 0, 2026-10-22. **Experimental source implementation exists; not independently audited or production-qualified.** See [implementation and validation](CONCEALED-PROTOCOL-IMPLEMENTATION.md). This document turns the agreements in [CONCEALED-PROTOCOL-DESIGN.md](CONCEALED-PROTOCOL-DESIGN.md) into a concrete candidate suite for Q2. Design references D1–D14 and Q1–Q14 retain their original meanings. A changed byte layout, primitive, or derivation requires a new suite; ordinary tuning must not silently change the wire format.
## 1. Scope and notation
The suite identifier is `1`, internal to authenticated encryption. This is a concealed connection flavor, not a renaming of released protocol 2. Endpoints explicitly select this flavor; no autodetection, downgrade, or TypeScript client change. TCP and WebSocket use stream records; UDP uses independent records. WebSocket message boundaries carry no additional protocol meaning: concatenate binary message payloads into the ordered byte stream. Reject text messages.
Integers are unsigned, fixed-width big-endian, except curve encodings, which follow their primitive's standard little-endian representation. `u8`, `u16`, `u32`, `u64` denote encoded widths. `a || b` denotes exact byte concatenation. `V(x) = u32(length(x)) || x`. No implicit text normalization, serializer metadata, native integer encoding, or trailing bytes are allowed. Text constants use ASCII. A label is `V(ASCII(label))`.
`H(x)` is BLAKE2b with output length **32**, no key. `B(n,k,x)` is BLAKE2b with configured output length `n` and key `k`; it is not truncation of a differently configured digest. Define:
```
Derive(n, secret, label, context) =
B(n, secret, V("kiloparsec/concealed/1") || V(label) || V(context))
```
Every input called `secret` below is exactly 32 bytes, except the explicitly 64-byte ordered directional-key material in token derivation. Keys never cross roles. Context and complete transcript hashes are public bindings, not entropy sources. Constant-time comparison is required for tags and secret-derived proof values. All randomness comes from libsodium's CSPRNG.
## 2. Suite and provisioned credentials
| Purpose | Primitive |
| --- | --- |
| Bootstrap agreement | X25519 with Elligator 2 concealed ephemeral public representation |
| Session agreement | libsodium `crypto_kx` (X25519/BLAKE2b), separate fresh key pairs |
| Bootstrap encryption and UDP | XSalsa20-Poly1305 `crypto_secretbox_easy` |
| Continuous stream encryption | Original ChaCha20, 32-byte key, 8-byte nonce, 64-bit block counter |
| Stream object authentication | Keyed BLAKE2b with 16-byte output |
| Hash/KDF | BLAKE2b as defined above |
| Identity signature | Ed25519 detached signature, 32-byte public key, 64-byte signature |
This first suite supports Ed25519 identity keys only. Other crypto2 key formats need another authenticated suite/identity encoding; do not silently serialize arbitrary polymorphic key objects here.
Client configuration contains an authentically obtained server static X25519 encryption public key `E` and trusted server Ed25519 verifying key `Vs`, with their association governed by application policy. The server owns corresponding private keys. No wire key identifier or automatic rotation protocol is present. Distinct configured endpoints can serve distinct credential sets. Future support for multiple retained keys must impose a small configured trial bound.
Original ChaCha20 is chosen because the cached common libsodium binding exposes it, including the 64-bit `initialCounter`; its XChaCha20 stream functions are commented out for Apple. This is a concrete candidate primitive under D11, not a claim that the complete composition has received independent review.
## 3. Elligator interoperability profile: qualification prerequisite
Let `b` be a fresh clamped X25519 scalar. Generate a Curve25519 public point including a uniformly randomized small-order component before inverse Elligator mapping. Retry with fresh local material if no representative exists. Encode the successful representative `R` as exactly 32 bytes with the otherwise unused high-bit choices randomized. Decode `R` to Montgomery public coordinate `P`. The server's static scalar is clamped, so its multiplication removes the small-order component. Preserve the secret scalar associated with `P` until bootstrap completion.
The Kotlin prototype now implements the [elligator.org Curve25519 profile](https://elligator.org/map), with `p = 2^255-19`, `A = 486662`, `Z = 2`. Square roots select the integer representative in `[0,(p-1)/2]`. Two most-significant serialized bits are independent random padding and are masked on decode. A separate random bit selects the inverse-map branch corresponding to the point's v sign. The full-curve public point is `clamp(b)*G + (b mod 8)*H`; the order-eight Edwards generator H has y coordinate `2707385501144840649318225287225658788936804267575313519463743609750303402022` and even x. Convert Edwards y to Montgomery `u=(1+y)/(1-y)` modulo p.
Published direct, inverse, and special scalar-multiplication vectors from [elligator.org/vectors](https://elligator.org/vectors/) are checked into `ElligatorReferenceVectors.kt` (retrieved 2026-10-22; initial fixture SHA-256 `d4518186b46e58476c2bfd5a3a6cd36f413cd8d969384d7cc59b4843651d85a3`). Tests cover all eight torsion components and independent libsodium agreement. This pins the prototype's profile; it does not establish all implementation properties. Expand to the extended vector/randomized corpus before qualification. Do not claim byte-for-byte compatibility with a particular Monocypher release without checking it separately.
Every use of a received X25519 public input checks its exact 32-byte width, propagates native rejection, and rejects an all-zero shared result before KDF. Do not regenerate local keys in response to a rejected peer key. The bignum mapping is variable-time, but the current call graph supplies public coordinates rather than the server static scalar. See the [timing assessment](CONCEALED-PROTOCOL-TIMING.md) for measurements, dataflow limits and response-shaping experiments; no constant-time or general timing-concealment guarantee is made.
## 4. Common protected bootstrap envelope
The first two flights use separately authenticated hidden lengths. This avoids a visible length field and avoids general object parsing before authentication.
Given a 32-byte root `K`, a context `C`, and a payload `M`:
```
Kh = Derive(32, K, "flight/header-key", C)
Kb = Derive(32, K, "flight/body-key", C)
Nh = Derive(24, K, "flight/header-iv", C)
Nb = Derive(24, K, "flight/body-iv", C)
L = u32(length(M))
Header = SecretBoxEasy(L, Nh, Kh) // 16-byte tag || 4 ciphertext bytes
Body = SecretBoxEasy(L || M, Nb, Kb) // 16-byte tag || ciphertext
Envelope(K,C,M) = Header || Body
```
`SecretBoxEasy` has exactly the libsodium combined layout. Distinct keys and IV derivations separate header/body operations. Duplicating `L` inside the body binds header and body; compare exact copies after both authenticate. Authenticate the 20-byte header before allocating the bounded body. Reject over-limit lengths, invalid lengths, failed authentication, and extra bytes within a datagram. Envelope payload length includes random padding. A stream reads precisely `20 + 16 + 4 + length(M)` bytes. The maximum flight includes its exposed prefix and all envelope overhead.
For UDP an envelope occupies one datagram. While waiting for a flight, silently discard undersized/oversized or unauthenticated datagrams and continue within the original attempt deadline. In particular, a valid authenticated header followed by a corrupted body must not consume the pending handshake state. Only a completely authenticated envelope with invalid semantic fields/signature makes the attempt terminal. TCP authentication failure remains terminal because there is no safe stream resynchronization. No preliminary error response, parsing oracle, or unauthenticated error record is sent. Invalid unauthenticated input is silently dropped; TCP may close without sending bytes. Enforce admission budgets before expensive key agreement, and whole-attempt/read deadlines throughout.
## 5. Client HELO, flight 1
Generate separate fresh bootstrap scalar `b` and fresh session `crypto_kx` pair `(c,C)`. They must not be reused across attempts, and must not be the identity signing key. `C` appears only inside the encrypted HELO.
```
Zb = X25519(b, E) // server computes X25519(e, P)
U = Random[24]
BootContext = R || U || E
Kboot = Derive(32, Zb, "bootstrap", BootContext)
HeloCore =
u8(1) // suite
|| u8(carrier) // 1 = stream, 2 = UDP
|| C[32]
|| Vs[32] // intended server identity
|| u32(receiveBodyLimit)
|| u16(receiveDatagramLimit) // 0 for stream; includes full datagram
|| u32(paddingMaximum)
HeloPayload = u32(length(HeloCore)) || HeloCore || randomPadding
F1 = R[32] || U[24] || Envelope(Kboot, H(BootContext), HeloPayload)
h1 = H(V("helo") || V(F1))
```
HeloCore is exactly **76 bytes**; F1 is **176 bytes plus padding**. All fields are fixed for suite 1; reject an unknown suite/carrier or a carrier inconsistent with this endpoint. No feature bitmask with undefined semantics is accepted. A server compares the intended `Vs` to its configured identity. Local limits constrain parsing before any peer-advertised limit. The inner core length must equal the suite-1 core size; the remaining bytes are padding.
Bind the exact representative bytes into the KDF, including bits ignored by the map. Equivalent representatives must not let an attacker mutate an accepted flight. The optional recent-client-session-DH ring guard atomically checks/inserts `C` after successful decryption and validation; duplicates are silently ignored. The guard is operational only and need not persist across restart.
## 6. Server hello, flight 2
After accepting HELO, generate fresh session `crypto_kx` pair `(s,S)` and random reply IV `I[24]`. The client session public key `C` is confidential bootstrap material here. Domain-separated derivation from it is the Q1 decision; it is not a general-purpose KDF from public information.
```
Kreply = Derive(32, C, "server-reply", h1 || E || Vs)
ReplyContext = H(V("server-flight") || h1 || I)
Proof = H(V("helo-decrypted") || C)
ServerCore =
u8(1) || u8(carrier)
|| S[32] || Proof[32] || Vs[32]
|| u32(serverReceiveBodyLimit)
|| u16(serverReceiveDatagramLimit)
|| u32(serverPaddingMaximum)
ServerSignature = Ed25519Sign(serverSigningKey,
V("kiloparsec/concealed/1/server-auth") || h1 || V(ServerCore))
ServerPayload = u32(length(ServerCore)) || ServerCore
|| ServerSignature[64] || randomPadding
F2 = I[24] || Envelope(Kreply, ReplyContext, ServerPayload)
h2 = H(V("handshake") || V(F1) || V(F2))
```
ServerCore is exactly **108 bytes**; F2 is **240 bytes plus padding**. The client authenticates hidden length/body, verifies suite/carrier, proof, intended server key, and signature before completing session derivation. The explicit proof is retained as agreed although authenticated response decryption already supplies a related proof. The server signature covers selected parameters, both session keys indirectly (`C` through exact F1), intended static identity through F1, and the response role.
Effective outbound bounds are the smaller relevant local/peer receive bounds. Peer claims can reduce a sender's allowance; they cannot increase a local allocation cap. The server may reduce its own padding; it must not require padding or silently change algorithms. UDP F2 must fit both configured datagram budgets and pre-validation amplification allowance; pad F1 sufficiently or reject locally before sending an unsupported attempt. Before valid ACK, server bytes sent must not exceed three times bytes received for the attempt.
A new reception of an old HELO after guard eviction generates fresh S and I. Random I prevents reply key/IV reuse. It does not turn the old hello into an authenticated client.
## 7. Session material, token, and handshake keys
Use the ordinary fresh session pairs through libsodium `crypto_kx`:
```
C2S = client.tx = server.rx // 32 bytes
S2C = client.rx = server.tx // 32 bytes
Token = Derive(32, C2S || S2C, "connection-token", h2)
```
Ordered directional key material, not process-local sending/receiving order, is essential. Do not substitute crypto2 `SessionKey.sessionTag` without checking its exact derivation against these bytes.
For each direction `d` in ASCII `c2s`, `s2c`, let `Kd` be the corresponding crypto_kx output. Define:
```
Kfinish[d] = Derive(32, Kd, "finished/" || d, h2)
```
ACK and acceptance use independent secretbox records even on TCP. They do not consume the subsequent continuous stream's cipher position. This makes handshake retransmission independent of stream state and avoids changing application traffic keys before both identities are bound.
## 8. Client ACK, flight 3
After server authentication, the client signs the session binding with its identity key. This suite supports an Ed25519 client identity `Vc`; application policy decides whether it is authorized. Optional anonymous identity requires a future explicit policy/specification; it is not an absent signature.
```
ClientSignature = Ed25519Sign(clientSigningKey,
V("kiloparsec/concealed/1/client-auth") || Token || h2 || Vc)
AckCore = u8(3) || h2[32] || Vc[32] || ClientSignature[64]
AckPayload = u32(length(AckCore)) || AckCore || randomPadding
A = Random[24]
F3 = A || Envelope(Kfinish[c2s], H(V("ack") || h2 || A), AckPayload)
h3 = H(V("authenticated-handshake") || h2 || V(F3))
```
AckCore is exactly **129 bytes**; F3 is **197 bytes plus padding**. A server accepts F3 only for the pending attempt associated with its listener and, for UDP, source address/port. It checks exact core size, role, transcript, signature, and authorization policy. No RPC is dispatched based on an unverified identity. No connected callback or authorization hook is repeated for a retransmitted identical authenticated F3.
F3 is retransmitted **byte-for-byte**, with the same A and ciphertext. Never regenerate a signature/padding/IV for a retry of this flight. Reusing an identical cached ciphertext is not encrypting a different plaintext with reused nonce material.
## 9. Server acceptance, flight 4
After successful client identity verification and application authorization:
```
AcceptCore = u8(4) || h3[32]
AcceptPayload = u32(length(AcceptCore)) || AcceptCore || randomPadding
J = Random[24]
F4 = J || Envelope(Kfinish[s2c], H(V("accept") || h2 || J), AcceptPayload)
```
AcceptCore is exactly **33 bytes**; F4 is **101 bytes plus padding**. There is no plaintext rejection variant. Failure aborts the attempt; authenticated diagnostic design is outside this initial suite. The client verifies F4 and its h3 before reporting connected. Cache exact F4 for the bounded UDP retry interval. Only a matching authenticated exact F3 may trigger its resend, within retry/rate budgets.
Identity-ready, authorized dispatch, and public connected state are separate lifecycle events internally. Existing onConnected policy handlers are synchronous KiloScope receiver functions, not suspending RPC callbacks. They inspect verified credentials before acceptance/readiness. An RPC launched asynchronously by a handler waits on the existing deferred readiness gate and runs after setup; a handler must not block waiting for it. Server policies run before F4; client policies run after F4 verification and before local readiness/connected notification. Preserve this existing initialization contract (D13/Q5), and test that background-launched calls release after readiness while rejecting policies fail queued calls. The handshake dispatcher remains separate from RPC.
## 10. Application traffic keys
Bind the client identity and exact ACK into traffic keys, avoiding unknown-peer associations across an identical DH transcript:
```
For each d:
KstreamBase[d] = Derive(32, Kd, "traffic/stream/" || d, h3)
StreamSalt[d] = Derive(24, Kd, "traffic/salt/" || d, h3)
Kmac[d] = Derive(32, Kd, "traffic/mac/" || d, h3)
Kudp[d] = Derive(32, Kd, "traffic/udp/" || d, h3)
```
The record engine expands the stream base/salt using exact ASCII labels (without V prefixes):
```
Kstream[d] = B(32, KstreamBase[d],
ASCII("kiloparsec/concealed/stream-key/v1") || StreamSalt[d])
Nstream[d] = first8(B(16, KstreamBase[d],
ASCII("kiloparsec/concealed/stream-nonce/v1") || StreamSalt[d]))
```
This two-stage derivation is the record engine's reusable API boundary; handshake-derived inputs bind direction and h3. BLAKE2b generic-hash output lengths are 16–64, so the 8-byte nonce is explicitly truncated from the 16-byte configured digest. h3 is available to both peers before acceptance; F4 confirms its agreement. Neither F4 randomness nor retransmission changes traffic keys.
Erase raw bootstrap/shared/finished material when it is no longer needed, subject to bounded retransmission state. JVM garbage collection limits guarantees of physical erasure; document actual platform behavior rather than promising complete erasure.
## 11. Protected record body
A record's authenticated body is:
```
u32(objectLength) || objectBytes || randomPadding
```
The first byte of objectBytes is the message kind: `0 = RPC`, `1 = CLOSE`. For RPC the remaining bytes are one existing BiPack-encoded `Transport.Block` with no existing L0Call encryption wrapper. Decode only the exact objectLength bytes after integrity verification; require complete consumption. CLOSE objectBytes is exactly `u8(1) || u16(reason)` with reason `0 = normal`, `1 = local shutdown`, `2 = usage limit`; reject undefined reasons in suite 1. An authenticated CLOSE conveys no success for pending RPCs.
Direction and transcript are bound through separately derived MAC keys; they need not be repeated in each MAC input. The authenticated body delimiter fixes the boundary between object and padding. Padding byte content is random, with an independent uniformly sampled length between zero and configured effective maximum. UDP reduces padding before rejecting an oversized unpadded record. No filler is appended outside authentication/encryption.
## 12. Stream application records
Maintain independent byte cursor and implicit record sequence q per direction, both starting at zero after F4. For body B:
```
L = u32(length(B))
T = B(16, Kmac[d], ASCII("kiloparsec/concealed/record-mac/v1")
|| u64(q) || L || B)
PlainRecord = L || B || T
WireRecord = PlainRecord XOR nextChaCha20Bytes(length(PlainRecord))
```
ChaCha20 begins at block counter zero with Kstream/Nstream. For byte cursor p, the relevant block is `floor(p/64)` and within-block offset is `p mod 64`. Socket segmentation must not reset the cursor or discard unused block bytes. A binding call with `initialCounter` starts at a block boundary: retain leftovers or generate/discard the prefix of that block correctly. Never restart the primitive per object with counter zero.
Decrypt exactly 4 header bytes; enforce local/effective body limit before allocation; read body/tag with partial-record deadline; recompute MAC using exact bytes and constant-time compare; only then parse the body and BiPack object. Increment q after successful authentication. Any bad tag, malformed authenticated record, over-limit length, partial-record timeout, or premature EOF fails the channel. Never scan for resynchronization.
A single writer selects encoded plaintext objects from OutQueue using admission ordinal and priority, assigns q in selected order, protects, and writes each record completely. Enqueue admission is atomic. Cancellation after stream mutation must finish that record or abort; it cannot roll back a cipher cursor. The existing queue's 2730-position bound supplies fairness; no null-ordinal normal entries bypass it. A large object is not preemptible after writing starts.
EOF following authenticated CLOSE is clean; EOF without CLOSE is abrupt/truncated. CLOSE follows drained admitted writes, with the configured drain deadline. No half-close API initially.
## 13. UDP application records
Each direction has a protected u64 sequence q starting at zero. Allocate sequence, encrypt, and enqueue atomically under the preparation mutex; await transmission outside it.
```
I = Random[24]
Plain = u64(q) || Body
Datagram = I || SecretBoxEasy(Plain, I, Kudp[d])
```
Overhead before object bytes/padding is 24 + 16 + 8 + 4 = **52 bytes**, including an object kind byte inside objectBytes makes RPC framing overhead 53 bytes before BiPack bytes. The encrypted inner delimiter and sequence are authenticated. Require the complete datagram within configured/negotiated budget. Verify secretbox before touching replay state or interpreting record fields. Reject malformed valid records; silently discard unauthenticated garbage without closing an established UDP session. Duplicate/stale authenticated data records are nonfatal discards. Replay tracking uses the existing 4096-position window; local overtaking is at most 2730.
CLOSE is sent once, best effort. Lost/reordered datagrams do not imply delivery of prior RPCs. No new application retransmission, fragmentation/reassembly, NAT rebinding, or session migration is provided. Keep one active authenticated association per source/listener under existing adapter ownership rules; fresh unverified HELOs cannot replace it.
Handshake retries remain separate from normal replay filtering. Pending state knows which flight it expects; acceptance cache recognizes exact authenticated F3 without rerunning callbacks. After establishment, process a possible retry as such only after verifying its matching finished key/context; do not reinterpret arbitrary application datagrams as RPC or mutate a live session based on unauthenticated classification. The absence of a visible session selector requires a strictly bounded number of candidate states per tuple. Initial implementation may permit one pending attempt and one established/retry state per tuple; retire superseded pending attempts without replacing the established one.
## 14. Limits and failures
Use the grouped immutable options and defaults from design section 6. Fixed security properties are not optional: authentication, trusted identity verification, transcript binding, failed-DH rejection, 16-byte tags, 24-byte independent-record IVs, and non-wrapping counters.
For this candidate suite, proactively reconnect before a direction exceeds **2^32 application records** or, for streams, **2^40 encrypted plaintext bytes** (including encrypted length and MAC). Datagram records are bounded to 65507 bytes each, so the 2^32-record ceiling implies fewer than 2^48 encrypted bytes; there is no additional cumulative UDP byte counter. Handshake keys are flight-specific and bounded by flight size/retry budgets. These are conservative selected engineering ceilings, not a security-bound proof. Check projected use before admitting a record; never wrap q, cursor, or cipher counter. Usage-limit CLOSE may use a reserved final allowance; otherwise abort. Do not reset counters under existing keys. No in-place rekey protocol in suite 1.
Bootstrap defaults: 15-second whole-attempt deadline, three fresh attempts, 16-KiB stream-flight cap, UDP cap 1200 bytes. F1/F2/F3/F4 must each fit its carrier cap after overhead. Every inner length is validated against its authenticated container. Stream body default 16 MiB, minimum body size 4 (a control/object decoder may impose a larger minimum); partial-record deadline 30 seconds beginning with the first byte, not while idle. Queue 128 objects/32 MiB, padding default 0–255, graceful close 2 seconds. Admission budgets, retransmission and cache bounds follow Q10/design section 6.
A local, server-only failure-work policy defaults to one discarded HELO decryption after envelope authentication fails, using a mutated copy of an independent throwaway scalar. It shares crypto admission bounds, never changes the real outcome or emits bytes, and can be disabled by `Handshake.decoyOnHelloAuthenticationFailure`. It changes no wire field and is not an additional authentication check. See the [timing assessment](CONCEALED-PROTOCOL-TIMING.md#7-requested-server-side-decoy-decryption).
A pre-authentication error produces no detailed wire diagnostic. Local typed exceptions retain safe diagnostic categories, excluding keys/plaintexts. No peer-controlled lengths or rate-table keys cause unbounded allocation. Application deadline overrides do not disable cryptographic checks.
## 15. Required vectors and implementation status
This file specifies candidate bytes and labels. Published Elligator vectors and [independent Python composition fixtures](CONCEALED-PROTOCOL-VECTORS.md) now cover exact F1–F4 bytes, signatures, role-assigned KX material, nonaligned continuous stream records and reordered UDP records. These passed on JVM, Kotlin/JS and Linux Native. Fixed F2–F4 fixtures compare production components; randomized integration tests exercise the complete state machines without a production entropy override. This is not a completed cryptographic review. Before freezing it:
1. Extend the pinned Elligator vector corpus and prove mixed-order agreement on every qualified platform; assess bignum timing and retained binary size.
2. Preserve and extend the independent F1–F4/KDF/signature/stream/UDP composition fixtures. A fully deterministic complete state-machine cross-implementation harness remains distinct from the completed component comparisons.
3. Verify malformed inputs, modified representative aliases, every authenticated length, incorrect roles/identities, low-order DH rejection, duplicate flights, replay eviction/restart, and key-usage boundaries.
4. Exercise initialization RPC readiness and authorization ordering against actual connection code before freezing integration behavior.
5. Independently review custom continuous encryption/MAC composition, KDF from concealed C, downgrade/identity binding, forward-secrecy scope, unauthenticated framing bounds, and rejection observability.
The first and second flights do not have forward secrecy against later compromise of server static encryption key e. Captured HELO then reveals C, which reveals the reply key and server hello. Fresh session DH still protects ACK/client identity and application traffic against that static-key compromise, assuming ephemeral secrets are erased and remain uncompromised. Server identity itself may already be publicly provisioned. No concealment guarantee covers traffic timing, endpoints, WebSocket setup, volume, or active probes possessing public server credentials.
## 16. Existing API evidence and primary references
Inspected crypto2 `SigningKey.sign`, `VerifyingKey.verify`, `SafeKeyExchange`, and `Hash`; common cached libsodium-binding 0.9.6 sources expose `GenericHash.genericHash(message, requestedHashLength, key)`, `SecretBox.easy/openEasy`, `ScalarMultiplication.scalarMultiplication`, and `Stream.chacha20XorIc(..., initialCounter: ULong, ...)`. High-level crypto2 default hash wrappers are not sufficient where this suite demands explicit output size/key. Exact per-platform behavior still needs runtime qualification.
- [Libsodium original ChaCha20 and counters](https://doc.libsodium.org/advanced/stream_ciphers/chacha20)
- [Libsodium generic hashing and explicit output size](https://doc.libsodium.org/hashing/generic_hashing)
- [Libsodium secretbox combined format](https://doc.libsodium.org/secret-key_cryptography/secretbox)
- [Libsodium crypto_kx roles and rejection](https://doc.libsodium.org/key_exchange)
- [Elligator key generation and subgroup randomization](https://elligator.org/key-exchange)
- [Monocypher reference Elligator API](https://monocypher.org/manual/elligator)
- [RFC 7748](https://www.rfc-editor.org/rfc/rfc7748.html)
### Primitive encoding check vectors
These small values were independently calculated using Python `hashlib.blake2b` (OpenSSL/Python implementation), not the Kotlin code. They validate selected byte encodings/KDF output parameters only; they are **not full handshake or Elligator vectors**.
```
secret/key = 000102030405060708090a0b0c0d0e0f
101112131415161718191a1b1c1d1e1f
context = a0a1a2a3
label = ASCII("vector")
Derive(32, secret, label, context) =
595859a9e444bb24d1df7c118eeef70fea86720249bd9292eba6e32744da6f19
Derive(24, secret, label, context) =
46d4b4de3827562b5654864913c2c46cb9802a6a71322599
record q = 7
body = 000000030161620000
L = 00000009
MAC using section12 exact domain and key above =
d0809fec300f48fb92d8eacd48284a8f
```
The record's objectBytes are `01 61 62`, with two zero padding bytes used solely for deterministic testing. Production padding is random. This vector treats object bytes opaquely and is not a valid RPC/control message example. A 24-byte BLAKE2b digest is not the first 24 bytes of its 32-byte digest; the first two vectors catch accidental truncation.
+19 -4
View File
@@ -1,9 +1,9 @@
# Kiloparsec # Kiloparsec
The Kotlin implementation and the native [TypeScript + WASM client](typescript/README.md) The Kotlin implementation and the native [TypeScript + WASM client](typescript/README.md)
are maintained here together. Current source version **0.8.0** uses are maintained here together. Current Kotlin source version **0.9.0-SNAPSHOT** retains
**protocol 2**, which authenticates RPC correlation metadata inside encrypted **protocol 2**, which authenticates RPC correlation metadata inside encrypted
packets. Both peers must be upgraded together; protocol 1 peers are rejected with packets, and adds an opt-in [concealed flavor](CONCEALED-PROTOCOL-IMPLEMENTATION.md). The lightweight TypeScript client remains at 0.8.0 and uses protocol 2. Both peers must be upgraded together; protocol 1 peers are rejected with
no automatic downgrade. See [the security analysis](typescript/docs/SECURITY.md). no automatic downgrade. See [the security analysis](typescript/docs/SECURITY.md).
The new generation of __PARanoid SECurity__ protocol, advanced, faster, more secure. It also allows connecting any " The new generation of __PARanoid SECurity__ protocol, advanced, faster, more secure. It also allows connecting any "
@@ -26,6 +26,8 @@ correlation vulnerability. Earlier 0.3–0.7 versions share the old envelope and
must not be assumed compatible with current source. Application command schemas must not be assumed compatible with current source. Application command schemas
and the Kotlin interface/session programming model remain unchanged. and the Kotlin interface/session programming model remain unchanged.
The 0.9.0-SNAPSHOT concealed flavor uses separate explicit endpoints and has no automatic downgrade or shared-port detection. It requires Kotlin 2.4 and `crypto2:0.9.6-SNAPSHOT`. See [release notes](RELEASE-0.9.0-SNAPSHOT.md) and [qualification evidence](CONCEALED-PROTOCOL-QUALIFICATION.md).
#### ID calculation algorithm is changed since 0.4.1 #### ID calculation algorithm is changed since 0.4.1
Historical note: 0.4.1 changed public/shared key ID derivation. Current deployments Historical note: 0.4.1 changed public/shared key ID derivation. Current deployments
@@ -74,11 +76,14 @@ The library should be used as a maven dependency, not as source.
### Declare maven repository: ### Declare maven repository:
Add the private repository to your `build.gradle.kts`, like: Declare the library and its existing backend repositories in `build.gradle.kts`:
```kotlin ```kotlin
repositories { repositories {
mavenCentral()
maven("https://maven.universablockchain.com/")
maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven") maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
maven("https://gitea.sergeych.net/api/packages/YoungBlood/maven")
} }
``` ```
@@ -89,7 +94,7 @@ It could be, depending on your project structure, something like:
```kotlin ```kotlin
val commonMain by getting { val commonMain by getting {
dependencies { dependencies {
api("net.sergeych:kiloparsec:0.6.8") api("net.sergeych:kiloparsec:0.9.0-SNAPSHOT")
} }
} }
``` ```
@@ -360,3 +365,13 @@ The script runs JVM and Kotlin/JS Node tests, TypeScript unit tests, then cross-
a disposable localhost JVM server built from this checkout. It does not publish a disposable localhost JVM server built from this checkout. It does not publish
or contact deployment hosts. Additional Gradle options can be passed to the script. or contact deployment hosts. Additional Gradle options can be passed to the script.
For TypeScript-only commands and examples, see [the module README](typescript/README.md). For TypeScript-only commands and examples, see [the module README](typescript/README.md).
## Experimental concealed protocol
A separate Kotlin Multiplatform flavor conceals the handshake and RPC metadata using
Elligator, authenticated encrypted records and configurable padding. It is under
active qualification and uses explicit endpoints. See the
[implementation guide](CONCEALED-PROTOCOL-IMPLEMENTATION.md),
[wire draft](CONCEALED-PROTOCOL-WIRE.md) and
[design decisions](CONCEALED-PROTOCOL-DESIGN.md). The existing protocol and lightweight
TypeScript client remain available.
+15
View File
@@ -0,0 +1,15 @@
# Kiloparsec 0.9.0-SNAPSHOT
This snapshot adds an opt-in Kotlin Multiplatform concealed transport flavor alongside the existing protocol 2. The lightweight TypeScript client remains at 0.8.0 and uses protocol 2. Existing endpoints do not switch flavors automatically.
The new flavor protects handshake and RPC metadata with an Elligator bootstrap, independently authenticated fresh session DH, encrypted records and configurable padding. Raw TCP, UDP and binary WebSocket carriers integrate with the existing RPC engine, initialization authorization hooks and bounded priority queue. UDP records decrypt independently and retain replay protection; TCP uses a continuous authenticated stream. Client establishment attempts are finite and do not automatically reissue application operations.
Server HELO authentication failures perform one discarded attempt with a mutated copy of a separate throwaway key, under the existing crypto admission limit. This configurable behavior cannot change authentication outcomes or send a response. It is supplementary failure-path work, without a general timing-concealment claim.
The existing transport's encrypted-response and complete-block decoding boundaries also use strict BiPack decoding, avoiding an empty-response/null hazard exposed by Native testing.
Validation passed: **127 JVM, 90 Kotlin/JS Node and 94 Linux Native tests**. This includes independent Python composition fixtures, 10,000 malformed record cases, 12,800 UDP records under impairment, bounded admission, 60 seconds of duplex/priority load, and actual TCP/UDP payload captures. Android, Wasm and all configured native compilation checks passed. See the [qualification record](CONCEALED-PROTOCOL-QUALIFICATION.md) for measurements, publication verification and precise coverage limits.
Use `net.sergeych:kiloparsec:0.9.0-SNAPSHOT` from the existing private Maven repository. The build requires Kotlin **2.4.0** and currently consumes `crypto2:0.9.6-SNAPSHOT`. No additional crypto runtime library was introduced. Initial API use requires `@OptIn(ExperimentalConcealedProtocol::class)`; [examples and configuration](CONCEALED-PROTOCOL-IMPLEMENTATION.md) are available.
SNAPSHOT was selected because this is the first distribution of the concealed wire flavor, it consumes a snapshot dependency, and browser/mobile/Apple runtimes have not been qualified. Independent fixtures verify our composition using trusted primitives; they do not constitute an external cryptographic audit or prove traffic indistinguishability. The API and concealed wire format may still change during qualification. Bootstrap-private-key compromise exposes the initial flights, while the separate authenticated fresh session DH protects client identity and application traffic, subject to ephemeral-secret erasure.
+2
View File
@@ -4,6 +4,8 @@ Reviewed commit: `237d404bcd4d52e015b5e0d7cd4feaec4bd848ff` — Add authenticate
Review date: 2026-10-21. Baseline: parent commit `88ceeb2`. Review date: 2026-10-21. Baseline: parent commit `88ceeb2`.
Follow-up design: [Concealed protocol design record](CONCEALED-PROTOCOL-DESIGN.md), including agreed decisions D1–D14, corrections C1–C8, and open questions Q1–Q14. This is separate future work; the review issues below retain their existing status.
This document records findings for separate discussion and repair. Issue numbers are stable. Implementation changes and their validation are recorded under closed issues. Proposed remedies for open issues are discussion options, not an agreed redesign. This document records findings for separate discussion and repair. Issue numbers are stable. Implementation changes and their validation are recorded under closed issues. Proposed remedies for open issues are discussion options, not an agreed redesign.
## Assessment ## Assessment
+27 -5
View File
@@ -9,17 +9,18 @@
*/ */
import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
import org.jetbrains.kotlin.gradle.plugin.mpp.NativeBuildType
plugins { plugins {
kotlin("multiplatform") version "2.2.21" kotlin("multiplatform") version "2.4.0"
id("org.jetbrains.kotlin.plugin.serialization") version "2.2.21" id("org.jetbrains.kotlin.plugin.serialization") version "2.4.0"
id("com.android.library") version "8.7.2" apply true id("com.android.library") version "8.7.2" apply true
`maven-publish` `maven-publish`
id("org.jetbrains.dokka") version "2.2.0" id("org.jetbrains.dokka") version "2.2.0"
} }
group = "net.sergeych" group = "net.sergeych"
version = "0.8.0" version = "0.9.0-SNAPSHOT"
repositories { repositories {
google() google()
@@ -40,7 +41,15 @@ kotlin {
testTask { useMocha { timeout = "30s" } } testTask { useMocha { timeout = "30s" } }
} }
} }
linuxX64() linuxX64 {
// Explicit qualification binary: do not add optimized test linking to ordinary builds.
if (providers.gradleProperty("concealedNativeBenchmark").map(String::toBoolean).getOrElse(false)) {
binaries.test("concealedBenchmark", listOf(NativeBuildType.RELEASE))
testRuns.create("concealedBenchmark") {
setExecutionSourceFrom(binaries.getTest("concealedBenchmark", NativeBuildType.RELEASE))
}
}
}
linuxArm64() linuxArm64()
macosArm64() macosArm64()
@@ -75,7 +84,9 @@ kotlin {
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2") implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.10.2")
implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.9.0") implementation("org.jetbrains.kotlinx:kotlinx-serialization-json:1.9.0")
api("io.ktor:ktor-client-core:$ktor_version") api("io.ktor:ktor-client-core:$ktor_version")
api("net.sergeych:crypto2:0.9.0") api("net.sergeych:crypto2:0.9.6-SNAPSHOT")
// Existing crypto backend, directly used by the experimental concealed channel.
implementation("net.sergeych:multiplatform-crypto-libsodium-bindings:0.9.6")
} }
} }
val androidMain by getting { val androidMain by getting {
@@ -204,3 +215,14 @@ tasks.register<Exec>("typescriptTest") {
workingDir(layout.projectDirectory.dir("typescript")) workingDir(layout.projectDirectory.dir("typescript"))
commandLine("npm", "test") commandLine("npm", "test")
} }
// The current libsodium binding captures _sodium_init during module loading.
// 0.7.16 exports it only after ready; pin the compatible transitive JS/Wasm runtime.
rootProject.plugins.withType<org.jetbrains.kotlin.gradle.targets.js.yarn.YarnPlugin> {
rootProject.the<org.jetbrains.kotlin.gradle.targets.js.yarn.YarnRootExtension>()
.resolution("libsodium-sumo", "0.7.15")
}
rootProject.plugins.withType<org.jetbrains.kotlin.gradle.targets.wasm.yarn.WasmYarnPlugin> {
rootProject.the<org.jetbrains.kotlin.gradle.targets.wasm.yarn.WasmYarnRootExtension>()
.resolution("libsodium-sumo", "0.7.15")
}
@@ -0,0 +1,12 @@
package net.sergeych.kiloparsec
/** Complete-block boundary used only after a concealed channel has authenticated its peer. */
internal interface ProtectedRpcDevice : Transport.Device {
val maxQueuedItems: Int
val maxQueuedBytes: Int
val maxOvertaking: Int
/** Called under Transport's admission mutex. Stream protection itself happens in send(). */
fun prepareBlock(bytes: UByteArray): OutgoingData
/** Called after writer shutdown while admission is locked. */
fun destroyProtection()
}
@@ -194,7 +194,11 @@ class Transport<S>(
readResponse: (UByteArray) -> UByteArray = { it }, readResponse: (UByteArray) -> UByteArray = { it },
build: (UInt) -> OutgoingData, build: (UInt) -> OutgoingData,
): UByteArray = sendBoundCallBlock(cmd.name, { raw -> ): UByteArray = sendBoundCallBlock(cmd.name, { raw ->
try { readResponse(unpack(cmd.resultSerializer, raw)) } try {
// utools.unpack treats empty bytes as null for Unit interoperability. Here the
// result is a non-null encoded byte array; a null-backed UByteArray can crash Native.
readResponse(net.sergeych.bipack.BipackDecoder.decode(cmd.resultSerializer, raw.asByteArray()))
}
catch (e: DiscardPacket) { throw e } catch (e: DiscardPacket) { throw e }
catch (e: Throwable) { throw RemoteInterface.SecurityException("Invalid authenticated response") } catch (e: Throwable) { throw RemoteInterface.SecurityException("Invalid authenticated response") }
}) { id -> }) { id ->
@@ -214,12 +218,25 @@ class Transport<S>(
val writer = writerReady.await() val writer = writerReady.await()
val awaitSent = outgoing.withLock { val awaitSent = outgoing.withLock {
if (isClosed) throw RemoteInterface.ClosedException() if (isClosed) throw RemoteInterface.ClosedException()
val frame = prepare() val raw = prepare()
writer.enqueue(frame.bytes, frame.nonce) val priority = net.sergeych.kiloparsec.adapter.blockPriority(raw.bytes)
val frame = (device as? ProtectedRpcDevice)?.prepareBlock(raw.bytes) ?: raw
writer.enqueue(frame.bytes, frame.nonce, priority)
} }
awaitSent() awaitSent()
} }
/** Freeze new admission before draining; used by authenticated channel shutdown. */
internal suspend fun drainForClose() {
outgoing.withLock { isClosed = true }
writerReady.await().flush()
}
internal suspend fun destroyProtection() = outgoing.withLock {
isClosed = true
(device as? ProtectedRpcDevice)?.destroyProtection()
}
internal suspend fun abortSecurity() { internal suspend fun abortSecurity() {
isClosed = true isClosed = true
device.close() device.close()
@@ -248,14 +265,19 @@ class Transport<S>(
coroutineScope { coroutineScope {
// One ordered submission to the carrier at a time: adapter queues cannot // One ordered submission to the carrier at a time: adapter queues cannot
// undo the nonce-aware scheduling performed here. // undo the nonce-aware scheduling performed here.
val writer = PriorityBlockWriter(this, continueOnWriteFailure = true) { device.send(it) } val protected = device as? ProtectedRpcDevice
val writer = PriorityBlockWriter(this,
maxQueuedItems = protected?.maxQueuedItems ?: 128,
maxQueuedBytes = protected?.maxQueuedBytes ?: (32 * 1024 * 1024),
maxOvertaking = protected?.maxOvertaking ?: MAX_NONCE_OVERTAKING,
continueOnWriteFailure = protected == null) { device.send(it) }
writerReady.complete(writer) writerReady.complete(writer)
debug { "awaiting incoming blocks" } debug { "awaiting incoming blocks" }
// todo: rewrite it to close the job with no exceptions at all, always // todo: rewrite it to close the job with no exceptions at all, always
while (isActive && !isClosed) { while (isActive && !isClosed) {
try { try {
device.input.receive().let { packed -> device.input.receive().let { packed ->
val b = unpack<Block>(packed) val b = net.sergeych.bipack.BipackDecoder.decode<Block>(packed.asByteArray())
when (b) { when (b) {
is Block.Error -> access.withLock { is Block.Error -> access.withLock {
val error = if (responseReaders.containsKey(b.forId)) UnauthenticatedOuterError() else localInterface.decodeError(b) val error = if (responseReaders.containsKey(b.forId)) UnauthenticatedOuterError() else localInterface.decodeError(b)
@@ -2,7 +2,7 @@ package net.sergeych.kiloparsec.adapter
import net.sergeych.kiloparsec.MAX_NONCE_OVERTAKING import net.sergeych.kiloparsec.MAX_NONCE_OVERTAKING
/** Externally synchronized. Nonces belong to one sending-key stream. */ /** Externally synchronized. Ordering values are per-stream nonces or local admission ordinals. */
internal class OutQueue<T>(private val allowance: ULong = MAX_NONCE_OVERTAKING.toULong()) { internal class OutQueue<T>(private val allowance: ULong = MAX_NONCE_OVERTAKING.toULong()) {
private data class Entry<T>(val value: T, val nonce: ULong?, val priority: Int) private data class Entry<T>(val value: T, val nonce: ULong?, val priority: Int)
private val entries = mutableListOf<Entry<T>>() private val entries = mutableListOf<Entry<T>>()
@@ -51,12 +51,13 @@ internal class PriorityBlockWriter(
private val maxQueuedItems: Int = 128, private val maxQueuedItems: Int = 128,
private val maxQueuedBytes: Int = 32 * 1024 * 1024, private val maxQueuedBytes: Int = 32 * 1024 * 1024,
private val continueOnWriteFailure: Boolean = false, private val continueOnWriteFailure: Boolean = false,
maxOvertaking: Int = net.sergeych.kiloparsec.MAX_NONCE_OVERTAKING,
private val write: suspend (UByteArray) -> Unit, private val write: suspend (UByteArray) -> Unit,
) { ) {
private class Pending(val bytes: UByteArray, val priority: Int, val done: CompletableDeferred<Unit>) private class Pending(val bytes: UByteArray, val priority: Int, val done: CompletableDeferred<Unit>)
private val mutex = Mutex() private val mutex = Mutex()
private val wake = Channel<Unit>(Channel.CONFLATED) private val wake = Channel<Unit>(Channel.CONFLATED)
private val pending = OutQueue<Pending>() private val pending = OutQueue<Pending>(maxOvertaking.toULong())
private var closed = false private var closed = false
private var writing: Pending? = null private var writing: Pending? = null
private val job = scope.launch { private val job = scope.launch {
@@ -94,9 +95,9 @@ internal class PriorityBlockWriter(
} }
/** Admission never waits for the carrier. Caller may hold its preparation mutex here. */ /** Admission never waits for the carrier. Caller may hold its preparation mutex here. */
suspend fun enqueue(bytes: UByteArray, nonce: ULong? = null): suspend () -> Unit { suspend fun enqueue(bytes: UByteArray, nonce: ULong? = null, priority: Int = blockPriority(bytes)): suspend () -> Unit {
currentCoroutineContext().ensureActive() currentCoroutineContext().ensureActive()
val entry = Pending(bytes, blockPriority(bytes), CompletableDeferred()) val entry = Pending(bytes, priority, CompletableDeferred())
mutex.withLock { mutex.withLock {
if (closed || !job.isActive) throw RemoteInterface.ClosedException() if (closed || !job.isActive) throw RemoteInterface.ClosedException()
pending.removeAll { !it.done.isActive } pending.removeAll { !it.done.isActive }
@@ -0,0 +1,109 @@
@file:OptIn(ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.crypto2.EncryptingPublicKey
import net.sergeych.crypto2.SigningKey
import net.sergeych.crypto2.VerifyingPublicKey
import net.sergeych.kiloparsec.*
import net.sergeych.kiloparsec.concealed.handshake.HandshakeFailure
import kotlin.time.TimeSource
/**
* One client establishment, with finite retries using a fresh carrier and DH keys per attempt.
* Once connected, carrier loss ends this run. Application calls are never silently replayed.
*/
@ExperimentalConcealedProtocol
class ConcealedClient<S>(
private val local: KiloInterface<S>,
private val session: S,
private val serverEncryptionKey: EncryptingPublicKey,
private val trustedServerIdentity: VerifyingPublicKey,
private val clientIdentity: SigningKey,
val options: ConcealedOptions = ConcealedOptions(),
private val connect: suspend () -> ConcealedCarrier,
) : RemoteInterface {
private val ready = CompletableDeferred<ConcealedConnection<S>>()
private var connection: ConcealedConnection<S>? = null
private var runJob: Job? = null
private val runAccess = Mutex()
private var used = false
private var stopped = false
suspend fun awaitReady() { ready.await().awaitReady() }
suspend fun remoteId() = ready.await().remoteId()
suspend fun token() = ready.await().token()
override suspend fun <A, R> call(cmd: Command<A, R>, args: A): R = ready.await().call(cmd, args)
override suspend fun <A> push(cmd: Command<A, Unit>, args: A) = ready.await().push(cmd, args)
suspend fun run(onConnectedStateChanged: ((Boolean) -> Unit)? = null): Unit = coroutineScope {
runAccess.withLock { check(!used) { "Run a client only once" }; used = true }
runJob = currentCoroutineContext()[Job]
// Avoid Native narrowing a mutable Throwable to the concrete fallback exception type.
var last: Throwable? = null
try {
repeat(options.handshake.maxAttempts) { index ->
if (stopped) throw RemoteInterface.ClosedException()
currentCoroutineContext().ensureActive()
val started = TimeSource.Monotonic.markNow()
var carrier: ConcealedCarrier? = null
var established = false
try {
carrier = withTimeout(options.handshake.timeoutMillis) { connect() }
val remaining = options.handshake.timeoutMillis - started.elapsedNow().inWholeMilliseconds
if (remaining <= 0) throw RemoteInterface.ClosedException("Connection attempt deadline exceeded")
val c = ConcealedConnection.client(local, carrier, session, serverEncryptionKey,
trustedServerIdentity, clientIdentity,
options.copy(handshake = options.handshake.copy(timeoutMillis = remaining)))
connection = c
c.run { connected ->
if (connected) { established = true; ready.complete(c) }
onConnectedStateChanged?.invoke(connected)
}
// An established run is never restarted/replayed by this helper.
if (established) return@coroutineScope
last = c.terminalFailure ?: RemoteInterface.ClosedException()
} catch (e: TimeoutCancellationException) {
currentCoroutineContext().ensureActive()
last = e
} catch (e: CancellationException) { throw e }
catch (e: HandshakeFailure) { throw e }
catch (e: LocalInterface.BreakConnectionException) { throw e }
catch (e: Throwable) {
if (established) throw e
last = e
} finally {
withContext(NonCancellable) { carrier?.close() }
connection = null
}
if (index + 1 < options.handshake.maxAttempts) {
val base = options.handshake.retryDelayMillis
val cap = options.handshake.maxRetryDelayMillis
val factor = 1L shl minOf(index, 20)
val backoff = if (base > cap / factor) cap else minOf(cap, base * factor)
val jitter = 0.8 + kotlin.random.Random.nextDouble() * 0.4
delay((backoff * jitter).toLong())
}
}
throw last ?: RemoteInterface.ClosedException()
} catch (e: Throwable) {
ready.completeExceptionally(e)
throw e
} finally {
ready.completeExceptionally(last ?: RemoteInterface.ClosedException())
runJob = null
}
}
suspend fun close() {
stopped = true
try { connection?.close() }
finally {
ready.completeExceptionally(RemoteInterface.ClosedException())
// close during a connector/backoff must also release its pending coroutine.
runJob?.cancel(CancellationException("Concealed client closed"))
}
}
}
@@ -0,0 +1,219 @@
@file:OptIn(ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.Semaphore
import kotlinx.coroutines.sync.withLock
import net.sergeych.crypto2.*
import net.sergeych.kiloparsec.*
import net.sergeych.kiloparsec.concealed.handshake.*
/** Share one instance between a listener's connections to share duplicate/admission bounds. */
@ExperimentalConcealedProtocol
class ConcealedServerIdentity(
val encryptionKey: DecryptingSecretKey,
val signingKey: SigningKey,
recentDhCapacity: Int = 4096,
cryptoConcurrency: Int = 4,
) {
private val mutex = Mutex()
private val crypto = Semaphore(cryptoConcurrency.also { require(it > 0) })
private val helloFailureNoise = HelloFailureNoise()
private val recent = ArrayDeque<UByteArray>()
private val capacity = recentDhCapacity.also { require(it >= 0) }
init { require(encryptionKey.keyBytes.size == 32 && signingKey.verifyingKey.keyBytes.size == 32) }
internal suspend fun <T> cryptographicOperation(operation: () -> T): T {
if (!crypto.tryAcquire()) throw RemoteInterface.ClosedException("Handshake capacity reached")
try { return operation() } finally { crypto.release() }
}
internal fun <T> helloAuthentication(
frame: UByteArray, maximum: Int, headerOnly: Boolean, enabled: Boolean, operation: () -> T,
): T = if (!enabled) operation() else helloFailureNoise.run(operation) { decoy ->
ServerHandshake.discardHelloDecryption(frame, decoy, maximum, headerOnly)
}
internal suspend fun remember(key: UByteArray): Boolean = mutex.withLock {
if (capacity == 0) return@withLock true
if (recent.any { it.contentEquals(key) }) return@withLock false
if (recent.size == capacity) recent.removeFirst()
recent.addLast(key.copyOf())
true
}
}
/**
* Explicit experimental flavor over raw carriers. The original Kilo connections remain available.
* Run one coroutine per connection; calls made before readiness suspend and fail if setup fails.
*/
@ExperimentalConcealedProtocol
class ConcealedConnection<S> private constructor(
private val local: KiloInterface<S>,
private val carrier: ConcealedCarrier,
private val session: S,
private val clientKey: SigningKey?,
private val serverEncryption: EncryptingPublicKey?,
private val trustedServer: VerifyingPublicKey?,
private val serverIdentity: ConcealedServerIdentity?,
val options: ConcealedOptions,
) : RemoteInterface {
private val ready = CompletableDeferred<Transport<KiloScope<S>>>()
private val identity = CompletableDeferred<VerifyingPublicKey>()
private val token = CompletableDeferred<UByteArray>()
private val runAccess = Mutex()
private var device: ProtectedChannelDevice? = null
private var transport: Transport<KiloScope<S>>? = null
private var used = false
private var stopping = false
private var connectedScope: KiloScope<S>? = null
var terminalFailure: Throwable? = null
private set
suspend fun remoteId(): VerifyingPublicKey = identity.await()
suspend fun token(): UByteArray = token.await().copyOf()
suspend fun awaitReady() { ready.await() }
override suspend fun <A, R> call(cmd: Command<A, R>, args: A): R = ready.await().call(cmd, args)
override suspend fun <A> push(cmd: Command<A, Unit>, args: A) = ready.await().push(cmd, args)
suspend fun run(onConnectedStateChanged: ((Boolean) -> Unit)? = null) = coroutineScope {
runAccess.withLock { check(!used) { "Run a connection only once" }; used = true }
val inbox = CarrierInbox(carrier)
var material: HandshakeSession? = null
var clientHandshake: ClientHandshake? = null
var serverHandshake: ServerHandshake? = null
var rpcJob: Job? = null
var readJob: Job? = null
try {
if (stopping) throw RemoteInterface.ClosedException()
var ack: UByteArray? = null
var accepted: UByteArray? = null
withTimeout(options.handshake.timeoutMillis) {
if (serverIdentity == null) {
val h = ClientHandshake(serverEncryption!!.keyBytes, trustedServer!!, clientKey!!, carrier.kind, options)
clientHandshake = h
carrier.send(h.hello)
val (_, clientAck) = inbox.verifiedFlight(44, flightLimit(options, carrier.kind), h::serverHelloSize, h::receiveServerHello)
carrier.send(clientAck)
material = if (carrier.kind == ConcealedCarrier.Kind.STREAM) {
inbox.verifiedFlight(44, flightLimit(options, carrier.kind), h::acceptanceSize, h::receiveAcceptance).second
} else {
var result: HandshakeSession? = null
var retries = 0
while (result == null) {
val factor = 1L shl minOf(retries, 2)
val wait = if (options.udp.ackRetryMillis > 4_000 / factor) 4_000
else options.udp.ackRetryMillis * factor
result = withTimeoutOrNull(wait) {
inbox.verifiedFlight(44, flightLimit(options, carrier.kind), h::acceptanceSize, h::receiveAcceptance).second
}
if (result == null) {
if (retries++ >= options.udp.maxAckRetries) throw RemoteInterface.ClosedException("Handshake acceptance timed out")
carrier.send(clientAck)
}
}
result
}
} else {
val maximum = flightLimit(options, carrier.kind)
val (_, h) = inbox.verifiedFlight(76, maximum, { prefix ->
serverIdentity.cryptographicOperation {
serverIdentity.helloAuthentication(prefix, maximum, true, options.handshake.decoyOnHelloAuthenticationFailure) {
ServerHandshake.helloSize(prefix, serverIdentity.encryptionKey.keyBytes, maximum)
}
}
}) { helo ->
serverIdentity.cryptographicOperation {
serverIdentity.helloAuthentication(helo, maximum, false, options.handshake.decoyOnHelloAuthenticationFailure) {
ServerHandshake(serverIdentity.encryptionKey.keyBytes, serverIdentity.signingKey, carrier.kind, options, helo)
}
}
}
serverHandshake = h
if (!serverIdentity.remember(h.clientDhPublicKey)) {
// No response for repeated HELO; caller retries with fresh keys.
awaitCancellation()
}
carrier.send(h.response)
val (clientAck, _) = inbox.verifiedFlight(44, maximum, h::ackSize, h::verifyAck)
material = h.session
ack = clientAck
// Run existing policy hooks with verified credentials before acceptance.
val verified = createScope(h.session)
local.onConnectHandlers.invokeAll(verified)
connectedScope = verified
accepted = h.accept()
carrier.send(accepted)
}
}
val active = material!!
val scope = connectedScope ?: createScope(active)
val protected = ProtectedChannelDevice(carrier, inbox, active, options, ack, accepted)
device = protected
val raw = Transport(protected, local, scope)
transport = raw
readJob = protected.start(this)
rpcJob = launch { raw.run() }
if (serverIdentity == null) {
local.onConnectHandlers.invokeAll(scope)
connectedScope = scope
}
identity.complete(active.peerIdentity)
token.complete(active.token.copyOf())
ready.complete(raw)
onConnectedStateChanged?.invoke(true)
rpcJob.join()
terminalFailure = protected.failure
} catch (e: CancellationException) { throw e }
catch (e: Throwable) { terminalFailure = e; throw e }
finally {
val failure = terminalFailure ?: RemoteInterface.ClosedException()
ready.completeExceptionally(failure)
identity.completeExceptionally(failure)
token.completeExceptionally(failure)
withContext(NonCancellable) {
carrier.close()
rpcJob?.cancelAndJoin()
readJob?.cancelAndJoin()
transport?.destroyProtection()
}
clientHandshake?.destroy(); serverHandshake?.destroy()
material?.let { it.sending.destroy(); it.receiving.destroy(); it.token.fill(0u) }
connectedScope?.let { local.onDisconnectHandlers.invokeAll(it) }
onConnectedStateChanged?.invoke(false)
}
}
private fun createScope(material: HandshakeSession): KiloScope<S> = object : KiloScope<S> {
override val session = this@ConcealedConnection.session
override val remote: RemoteInterface = this@ConcealedConnection
override val sessionToken = material.token.copyOf()
override val remoteIdentity = material.peerIdentity
}
/** Drain admitted writes within a deadline, send authenticated CLOSE, then release all waiters. */
suspend fun close() = withContext(NonCancellable) {
stopping = true
try {
withTimeoutOrNull(options.close.timeoutMillis) {
transport?.drainForClose()
device?.sendClose()
}
} catch (_: Exception) { /* Graceful close is best effort; always release the carrier. */ }
finally { carrier.close() }
ready.completeExceptionally(RemoteInterface.ClosedException())
identity.completeExceptionally(RemoteInterface.ClosedException())
token.completeExceptionally(RemoteInterface.ClosedException())
}
companion object {
fun <S> client(
local: KiloInterface<S>, carrier: ConcealedCarrier, session: S,
serverEncryptionKey: EncryptingPublicKey, trustedServerIdentity: VerifyingPublicKey,
clientIdentity: SigningKey, options: ConcealedOptions = ConcealedOptions(),
) = ConcealedConnection(local, carrier, session, clientIdentity, serverEncryptionKey, trustedServerIdentity, null, options)
fun <S> server(
local: KiloInterface<S>, carrier: ConcealedCarrier, session: S,
serverIdentity: ConcealedServerIdentity, options: ConcealedOptions = ConcealedOptions(),
) = ConcealedConnection(local, carrier, session, null, null, null, serverIdentity, options)
}
}
@@ -0,0 +1,78 @@
package net.sergeych.kiloparsec.concealed
/** Experimental wire protocol; compatibility and side-channel qualification are still in progress. */
@RequiresOptIn(level = RequiresOptIn.Level.WARNING)
@Retention(AnnotationRetention.BINARY)
annotation class ExperimentalConcealedProtocol
/** Tunable resource policies; cryptographic checks cannot be disabled. All sizes are bytes. */
@ExperimentalConcealedProtocol
data class ConcealedOptions(
val handshake: Handshake = Handshake(),
val limits: Limits = Limits(),
val padding: Padding = Padding(),
val queue: Queue = Queue(),
val udp: Udp = Udp(),
val close: Close = Close(),
) {
data class Handshake(
val timeoutMillis: Long = 15_000,
val maxAttempts: Int = 3,
val retryDelayMillis: Long = 700,
val maxRetryDelayMillis: Long = 5_000,
/** Server-only bounded decoy decrypt after failed HELO envelope authentication. */
val decoyOnHelloAuthenticationFailure: Boolean = true,
) {
init {
require(timeoutMillis > 0 && maxAttempts > 0)
require(retryDelayMillis >= 0 && maxRetryDelayMillis >= retryDelayMillis)
}
}
data class Limits(
val maxBootstrapBytes: Int = 16 * 1024,
val maxBodyBytes: Int = 16 * 1024 * 1024,
val partialRecordTimeoutMillis: Long = 30_000,
) {
init {
require(maxBootstrapBytes in 512..1024 * 1024)
require(maxBodyBytes in 256..Int.MAX_VALUE - 64)
require(partialRecordTimeoutMillis > 0)
}
}
data class Padding(val maxBytes: Int = 255) {
init { require(maxBytes in 0..1024 * 1024) }
}
data class Queue(
val maxItems: Int = 128,
val maxBytes: Int = 32 * 1024 * 1024,
val maxOvertaking: Int = 2730,
) {
init { require(maxItems > 0 && maxBytes > 0 && maxOvertaking in 0..2730) }
}
data class Udp(
val maxDatagramBytes: Int = 1200,
val ackRetryMillis: Long = 1_000,
val maxAckRetries: Int = 3,
val acceptanceCacheMillis: Long = 30_000,
) {
init {
require(maxDatagramBytes in 512..65_507)
require(ackRetryMillis > 0 && maxAckRetries in 0..10 && acceptanceCacheMillis > 0)
}
}
data class Close(val timeoutMillis: Long = 2_000) {
init { require(timeoutMillis > 0) }
}
}
/** Raw carrier: stream chunks need no boundaries; datagrams must retain theirs. No legacy framing. */
@ExperimentalConcealedProtocol
interface ConcealedCarrier {
val kind: Kind
enum class Kind { STREAM, DATAGRAM }
suspend fun receive(): UByteArray?
/** Returns after writing the bytes, or throws. Calls are serialized by the channel. */
suspend fun send(bytes: UByteArray)
/** Releases pending reads/writes; idempotent and must not throw. */
suspend fun close()
}
@@ -0,0 +1,126 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import io.ktor.client.HttpClient
import io.ktor.client.plugins.websocket.WebSockets
import io.ktor.client.plugins.websocket.webSocketSession
import io.ktor.http.Url
import io.ktor.websocket.Frame
import io.ktor.websocket.WebSocketSession
import kotlinx.coroutines.NonCancellable
import kotlinx.coroutines.cancel
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import kotlinx.coroutines.withContext
private const val DEFAULT_WEBSOCKET_CHUNK = 16 * 1024 * 1024 + 20
/**
* Explicit ws/wss endpoint carrying the concealed byte stream as binary frames.
* A supplied client remains caller-owned and must already install WebSockets. An omitted client
* has bounded frames, no periodic pinger, and is closed with the carrier or on connection failure.
* WebSocket framing remains visible; it is not part of the concealed protocol's wire encoding.
*/
@ExperimentalConcealedProtocol
suspend fun connectConcealedWebSocketCarrier(
url: String,
client: HttpClient? = null,
maxChunkBytes: Int = DEFAULT_WEBSOCKET_CHUNK,
): ConcealedCarrier {
require(maxChunkBytes > 0)
val parsed = Url(url)
require(parsed.protocol.name == "ws" || parsed.protocol.name == "wss")
val ownsClient = client == null
val actual = client ?: HttpClient {
install(WebSockets) {
maxFrameSize = maxChunkBytes.toLong()
pingIntervalMillis = 0
}
}
try {
val session = actual.webSocketSession(urlString = url)
return rawWebSocketCarrier(session, maxChunkBytes) { if (ownsClient) actual.close() }
} catch (e: Throwable) {
if (ownsClient) actual.close()
throw e
}
}
/**
* Wrap an existing client or server session. Configure its WebSockets plugin's frame bounds before
* accepting traffic too; this wrapper also lowers maxFrameSize and validates received binary chunks.
* No Base64, RPC framing, priority queue or keepalive mechanism is introduced.
*/
@ExperimentalConcealedProtocol
fun WebSocketSession.asConcealedCarrier(maxChunkBytes: Int = DEFAULT_WEBSOCKET_CHUNK): ConcealedCarrier =
rawWebSocketCarrier(this, maxChunkBytes)
private fun rawWebSocketCarrier(
session: WebSocketSession,
maxChunkBytes: Int,
onClose: () -> Unit = {},
): ConcealedCarrier {
require(maxChunkBytes > 0)
session.maxFrameSize = minOf(session.maxFrameSize, maxChunkBytes.toLong())
return object : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.STREAM
private val closeAccess = Mutex()
private val writeAccess = Mutex()
private var closed = false
override suspend fun receive(): UByteArray? {
try {
while (true) {
val result = session.incoming.receiveCatching()
val frame = result.getOrNull()
if (frame == null) {
result.exceptionOrNull()?.let { throw it }
close()
return null
}
when (frame) {
is Frame.Binary -> {
require(frame.data.size <= maxChunkBytes) { "WebSocket chunk exceeds configured bound" }
if (frame.data.isNotEmpty()) return frame.data.copyOf().asUByteArray()
}
is Frame.Text -> error("Concealed WebSocket requires binary frames")
is Frame.Close -> { close(); return null }
else -> Unit // Ping/pong handled by normal Ktor sessions.
}
}
} catch (e: Throwable) {
close()
throw e
}
}
override suspend fun send(bytes: UByteArray): Unit = writeAccess.withLock {
check(closeAccess.withLock { !closed }) { "Carrier is closed" }
try {
var offset = 0
while (offset < bytes.size) {
val end = offset + minOf(maxChunkBytes, bytes.size - offset)
session.send(Frame.Binary(true, bytes.copyOfRange(offset, end).asByteArray()))
session.flush()
offset = end
}
} catch (e: Throwable) {
close()
throw e
}
}
override suspend fun close(): Unit = withContext(NonCancellable) {
closeAccess.withLock {
if (!closed) {
closed = true
// Concealed CLOSE is sent by the protected channel. Do not wait indefinitely
// for a separate WebSocket close exchange or leave blocked reads/writes alive.
runCatching { session.cancel() }
runCatching { onClose() }
}
}
}
}
}
@@ -0,0 +1,42 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import net.sergeych.kiloparsec.concealed.handshake.HandshakeFailure
import net.sergeych.kiloparsec.concealed.handshake.UnauthenticatedFlight
/** Bounded extra work after failed HELO authentication; this is not constant-time padding. */
internal class HelloFailureNoise(
seed: UByteArray = LibsodiumRandom.buf(32),
private val random: (Int) -> UByteArray = LibsodiumRandom::buf,
) {
private val seed = seed.copyOf().also { require(it.size == 32) }
fun <T> run(real: () -> T, decoy: (UByteArray) -> Unit): T = try {
real()
} catch (original: UnauthenticatedFlight) {
val temporary = seed.copyOf()
try {
// The three lowest bits and highest two bits are clamped by X25519. Mutate
// only effective bits, without retries or duplicate positions.
val choices = IntArray(251) { it + 3 }
val entropy = random(17)
try {
val count = 1 + (entropy[0].toInt() and 7)
repeat(count) { i ->
val value = (entropy[1 + i * 2].toInt() shl 8) or entropy[2 + i * 2].toInt()
val selected = i + value % (choices.size - i)
val bit = choices[selected]
choices[selected] = choices[i]
choices[i] = bit
temporary[bit / 8] = (temporary[bit / 8].toInt() xor (1 shl (bit % 8))).toUByte()
}
} finally { entropy.fill(0u) }
try { decoy(temporary) } catch (_: HandshakeFailure) {
// Expected dummy authentication/parsing rejection cannot replace the real error.
}
} finally { temporary.fill(0u) }
throw original
}
}
@@ -0,0 +1,196 @@
@file:OptIn(ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.channels.SendChannel
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.kiloparsec.*
import net.sergeych.kiloparsec.concealed.handshake.HandshakeSession
import net.sergeych.kiloparsec.concealed.handshake.UnauthenticatedFlight
import net.sergeych.kiloparsec.concealed.record.*
/** Contains no RPC semantics beyond the authenticated internal DATA/CLOSE discriminant. */
internal class ProtectedChannelDevice(
private val carrier: ConcealedCarrier,
private val inbox: CarrierInbox,
session: HandshakeSession,
private val options: ConcealedOptions,
private val duplicateAck: UByteArray? = null,
private val acceptance: UByteArray? = null,
) : ProtectedRpcDevice {
private val incoming = Channel<UByteArray>(16)
override val input get() = incoming
// Transport uses send() exclusively; avoid an unsupervised parallel legacy write path.
override val output: SendChannel<UByteArray> = Channel<UByteArray>(0).also { it.close() }
override val maxQueuedItems get() = options.queue.maxItems
override val maxQueuedBytes get() = options.queue.maxBytes
override val maxOvertaking get() = options.queue.maxOvertaking
private val writes = Mutex()
private var closed = false
private var ordinal = 0uL
private val sendingLimits = session.sendingLimits
private val streamTx = if (carrier.kind == ConcealedCarrier.Kind.STREAM) session.sending.let {
TcpRecordEncoder(it.streamKey, it.macKey, it.streamSalt, session.sendingLimits)
} else null
private val streamRx = if (carrier.kind == ConcealedCarrier.Kind.STREAM) session.receiving.let {
TcpRecordDecoder(it.streamKey, it.macKey, it.streamSalt, session.receivingLimits)
} else null
private val udpTx = if (streamTx == null) UdpRecordEncoder(session.sending.udpKey, session.sendingLimits) else null
private val udpRx = if (streamRx == null) UdpRecordDecoder(session.receiving.udpKey, session.receivingLimits) else null
var failure: Throwable? = null
private set
var receivedClose = false
private set
override fun prepareBlock(bytes: UByteArray): OutgoingData {
check(!closed)
val body = ubyteArrayOf(0u) + bytes
return if (streamTx != null) {
require(body.size <= sendingLimits.maxBodyBytes - 4) { "Object exceeds record limit" }
check(ordinal != ULong.MAX_VALUE) { "Admission space exhausted" }
OutgoingData(body, ordinal++)
} else udpTx!!.encode(body).let { OutgoingData(it.bytes, it.sequence) }
}
override suspend fun send(block: UByteArray) = writes.withLock {
if (closed) throw RemoteInterface.ClosedException()
try { carrier.send(streamTx?.encode(block) ?: block) }
catch (e: Throwable) { closed = true; carrier.close(); throw e }
}
fun start(scope: CoroutineScope) = scope.launch {
var problem: Throwable? = null
var ackResponses = 0
val retryUntil = kotlin.time.TimeSource.Monotonic.markNow()
var partialSince: kotlin.time.TimeMark? = null
try {
while (isActive && !closed) {
val chunk = if (streamRx?.hasPartialRecord == true) {
val started = partialSince ?: kotlin.time.TimeSource.Monotonic.markNow().also { partialSince = it }
val remaining = options.limits.partialRecordTimeoutMillis - started.elapsedNow().inWholeMilliseconds
if (remaining <= 0) throw InvalidRecord("Partial record deadline exceeded")
withTimeout(remaining) { inbox.next() }
} else inbox.next()
val arrived = kotlin.time.TimeSource.Monotonic.markNow()
if (chunk == null) {
streamRx?.finish()
if (streamRx != null && !receivedClose) throw InvalidRecord("Stream ended without authenticated CLOSE")
break
}
if (udpRx != null && duplicateAck != null && acceptance != null &&
retryUntil.elapsedNow().inWholeMilliseconds < options.udp.acceptanceCacheMillis && ackResponses < options.udp.maxAckRetries &&
chunk.contentEquals(duplicateAck)) {
writes.withLock { if (!closed) carrier.send(acceptance) }
ackResponses++
continue
}
val records = if (streamRx != null) streamRx.feed(chunk) else {
val record = try { udpRx!!.decode(chunk) }
catch (_: UnauthenticatedDatagram) { continue }
listOfNotNull(record)
}
if (streamRx != null) {
if (!streamRx.hasPartialRecord) partialSince = null
else if (partialSince == null || records.isNotEmpty()) partialSince = arrived
}
for (record in records) {
if (record.isEmpty()) throw InvalidRecord("Missing channel record kind")
when (record[0].toInt()) {
0 -> {
if (receivedClose) throw InvalidRecord("Data after CLOSE")
incoming.send(record.copyOfRange(1, record.size))
}
1 -> {
if (record.size != 3 || record[1] != 0u.toUByte() || record[2].toInt() !in 0..2) throw InvalidRecord("Invalid CLOSE")
receivedClose = true
return@launch
}
else -> throw InvalidRecord("Unknown channel record kind")
}
}
}
} catch (_: TimeoutCancellationException) {
problem = InvalidRecord("Partial record deadline exceeded")
failure = problem
} catch (e: CancellationException) { throw e }
catch (e: Throwable) { problem = e; failure = e }
finally {
if (problem == null) incoming.close() else incoming.cancel(CancellationException("Protected input failed", problem))
closed = true
withContext(NonCancellable) { carrier.close() }
}
}
suspend fun sendClose() = writes.withLock {
if (!closed) {
val record = ubyteArrayOf(1u, 0u, 0u)
carrier.send(streamTx?.encode(record) ?: udpTx!!.encode(record).bytes)
}
}
override fun destroyProtection() {
streamTx?.destroy(); streamRx?.destroy(); udpTx?.destroy(); udpRx?.destroy()
}
override suspend fun close() {
closed = true
incoming.cancel()
carrier.close()
}
}
/** Keeps stream over-read bytes across handshake/data transitions; never guesses a boundary. */
internal class CarrierInbox(private val carrier: ConcealedCarrier) {
private var pending = ubyteArrayOf()
private var offset = 0
suspend fun next(): UByteArray? {
if (offset < pending.size) return pending.copyOfRange(offset, pending.size).also { offset = pending.size }
return carrier.receive()
}
private suspend fun exact(size: Int): UByteArray {
val result = UByteArray(size)
var used = 0
while (used < size) {
if (offset == pending.size) {
pending = carrier.receive() ?: throw RemoteInterface.ClosedException("Closed during handshake")
offset = 0
if (pending.isEmpty()) continue
}
val n = minOf(size - used, pending.size - offset)
pending.copyInto(result, used, offset, offset + n)
used += n; offset += n
}
return result
}
suspend fun <T> verifiedFlight(
prefixSize: Int, maximum: Int, size: suspend (UByteArray) -> Int,
verify: suspend (UByteArray) -> T,
): Pair<UByteArray, T> {
while (true) {
try {
val bytes = flight(prefixSize, maximum, size)
return bytes to verify(bytes)
} catch (e: UnauthenticatedFlight) {
if (carrier.kind != ConcealedCarrier.Kind.DATAGRAM) throw e
}
}
}
suspend fun flight(prefixSize: Int, maximum: Int, size: suspend (UByteArray) -> Int): UByteArray {
if (carrier.kind == ConcealedCarrier.Kind.DATAGRAM) {
while (true) {
val packet = next() ?: throw RemoteInterface.ClosedException("Closed during handshake")
if (packet.size !in prefixSize..maximum) continue
try {
if (size(packet.copyOfRange(0, prefixSize)) != packet.size) continue
return packet
} catch (_: UnauthenticatedFlight) { /* Drop unauthenticated noise until deadline. */ }
}
}
val prefix = exact(prefixSize)
val total = size(prefix)
require(total in prefixSize..maximum) { "Invalid handshake frame size" }
return prefix + exact(total - prefixSize)
}
}
@@ -0,0 +1,141 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.elligator
import com.ionspin.kotlin.bignum.integer.BigInteger
import com.ionspin.kotlin.bignum.integer.Sign
import com.ionspin.kotlin.crypto.ed25519.Ed25519LowLevel
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
/**
* Experimental Elligator 2 / Curve25519 implementation, Z=2.
*
* This readable implementation uses VARIABLE-TIME arbitrary-precision field arithmetic. It is
* intended for compatibility experiments, not a production timing-security claim. Initialise
* libsodium before calling [generateKeyPair] or [specialPublicKey]. The representative is an
* ephemeral, single-use wire value. Bind its EXACT bytes into the bootstrap authentication.
*
* Formulas and full-curve generation: https://elligator.org/map and
* https://elligator.org/key-exchange. Representatives use the nonnegative root (<= (p-1)/2),
* little endian, with the two high bits randomised and ignored by [decode].
*/
internal object ExperimentalElligator25519 {
private val zero = BigInteger.ZERO
private val one = BigInteger.ONE
private val two = BigInteger.fromInt(2)
private val p = (one shl 255) - BigInteger.fromInt(19)
private val a = BigInteger.fromInt(486662)
private val half = (p - one) / two
private val sqrtMinusOne = pow(two, (p - one) / BigInteger.fromInt(4))
class KeyPair internal constructor(val secretKey: UByteArray, val representative: UByteArray) {
/** Best effort: managed runtimes may retain copies. */
fun destroy() = secretKey.fill(0u)
}
fun generateKeyPair(): KeyPair {
while (true) {
val secret = LibsodiumRandom.buf(32)
try {
val encoded = encode(specialPublicKey(secret), LibsodiumRandom.buf(1)[0].toInt())
if (encoded != null) return KeyPair(secret, encoded)
} catch (failure: Throwable) {
secret.fill(0u)
throw failure
}
secret.fill(0u)
}
}
/** Full-curve public key: clamp(s)*G + (s mod 8)*H, converted to Montgomery u. */
fun specialPublicKey(secretKey: UByteArray): UByteArray {
require(secretKey.size == 32) { "An X25519 secret must contain 32 bytes" }
val high = Ed25519LowLevel.scalarMultiplicationBase(secretKey)
// The low scalar bits are independent of the clamped secret, which clears them.
// Constant-time table selection avoids an extra secret-dependent lookup. Field maths
// below remains variable time; this alone does not establish timing security.
val low = UByteArray(32)
val index = secretKey[0].toInt() and 7
for (i in 0..7) {
val mask = ((i xor index) - 1) shr 31
for (j in 0..31) low[j] = (low[j].toInt() or (torsion[i][j].toInt() and mask)).toUByte()
}
val edwards = Ed25519LowLevel.addPoints(high, low)
edwards[31] = (edwards[31].toInt() and 127).toUByte()
val y = read(edwards)
return write(mod((one + y) * inverse(one - y)))
}
/** Decode any 32-byte representative; low-order results must be rejected by DH agreement. */
fun decode(representative: UByteArray): UByteArray {
require(representative.size == 32) { "An Elligator representative must contain 32 bytes" }
val bytes = representative.copyOf()
bytes[31] = (bytes[31].toInt() and 63).toUByte()
val r = read(bytes)
val w = mod(-a * inverse(one + two * r * r))
val square = sqrt(mod(w * w * w + a * w * w + w)) != null
return write(if (square) w else mod(-w - a))
}
/**
* Inverse map of a canonical Montgomery u, or null if unrepresentable.
* Tweak bit 0 selects the v sign; bits 6 and 7 fill otherwise unused encoding bits.
* These bits must be random and independent of the low-order-point selection.
*/
fun encode(publicKey: UByteArray, tweak: Int): UByteArray? {
require(publicKey.size == 32) { "A Montgomery coordinate must contain 32 bytes" }
val u = read(publicKey)
require(u < p) { "Montgomery coordinate must be canonical" }
if (u == mod(-a)) return null
if (sqrt(mod(-two * u * (u + a))) == null) return null
val r = if (tweak and 1 == 0) {
sqrt(mod(-u * inverse(two * (u + a))))
} else {
if (u == zero) return null
sqrt(mod(-(u + a) * inverse(two * u)))
} ?: return null
return write(r).also { it[31] = (it[31].toInt() or (tweak and 192)).toUByte() }
}
private fun mod(n: BigInteger) = n mod p
private fun inverse(n: BigInteger): BigInteger {
val value = mod(n)
require(value != zero) { "Cannot invert zero" }
return value.modInverse(p)
}
private fun pow(value: BigInteger, exponent: BigInteger): BigInteger {
var result = one
var base = mod(value)
var e = exponent
while (e != zero) {
if (e.bitAt(0)) result = mod(result * base)
base = mod(base * base)
e = e shr 1
}
return result
}
private fun sqrt(value: BigInteger): BigInteger? {
val n = mod(value)
var r = pow(n, (p + BigInteger.fromInt(3)) / BigInteger.fromInt(8))
if (mod(r * r) != n) r = mod(r * sqrtMinusOne)
if (mod(r * r) != n) return null
return if (r > half) p - r else r
}
private fun read(bytes: UByteArray) = BigInteger.fromUByteArray(bytes.reversedArray(), Sign.POSITIVE)
private fun write(n: BigInteger): UByteArray {
val bytes = n.toUByteArray().reversedArray()
return UByteArray(32) { if (it < bytes.size) bytes[it] else 0u }
}
private val torsion: List<UByteArray> by lazy {
// Order-eight generator H from elligator.org/key-exchange; x is even, so sign bit=0.
val h = write(BigInteger.parseString(
"2707385501144840649318225287225658788936804267575313519463743609750303402022", 10))
val points = mutableListOf(UByteArray(32).also { it[0] = 1u })
repeat(7) { points += Ed25519LowLevel.addPoints(points.last(), h) }
points
}
}
@@ -0,0 +1,124 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.keyexchange.KeyExchange
import com.ionspin.kotlin.crypto.keyexchange.KeyExchangeSessionKeyPair
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import net.sergeych.crypto2.SigningKey
import net.sergeych.crypto2.VerifyingPublicKey
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import net.sergeych.kiloparsec.concealed.elligator.ExperimentalElligator25519
/** One fresh attempt; state mutation is externally serialized. No automatic downgrade/reuse. */
internal class ClientHandshake(
serverEncryptionPublic: UByteArray,
private val trustedServer: VerifyingPublicKey,
private val identity: SigningKey,
private val carrier: ConcealedCarrier.Kind,
private val options: ConcealedOptions = ConcealedOptions(),
) {
private val serverEncryptionPublic = serverEncryptionPublic.copyOf()
private val pair = KeyExchange.keypair()
private val local = PeerBounds.local(options, carrier)
private val maximum = flightLimit(options, carrier)
private var state = 0
val hello: UByteArray
private val h1: UByteArray
private var h2: UByteArray = ubyteArrayOf()
private var h3: UByteArray = ubyteArrayOf()
private lateinit var keys: KeyExchangeSessionKeyPair
private lateinit var peerBounds: PeerBounds
private lateinit var pendingSession: HandshakeSession
init {
require(serverEncryptionPublic.size == 32 && trustedServer.keyBytes.size == 32 && identity.verifyingKey.keyBytes.size == 32)
val bootstrap = ExperimentalElligator25519.generateKeyPair()
val shared = ScalarMultiplication.scalarMultiplication(bootstrap.secretKey, this.serverEncryptionPublic)
ensure(shared.any { it != 0u.toUByte() }, "Rejected bootstrap DH")
val iv = LibsodiumRandom.buf(24)
val context = bootstrap.representative + iv + this.serverEncryptionPublic
val root = derive(32, shared, "bootstrap", context)
try {
val core = ubyteArrayOf(1u, carrierByte(carrier)) + pair.publicKey + trustedServer.keyBytes + local.bytes()
hello = bootstrap.representative + iv + FlightEnvelope(root, hash(context), maximum - 56).seal(padded(core, options, maximum = maximum - 96))
} finally {
bootstrap.destroy(); shared.fill(0u); root.fill(0u)
}
h1 = hash(label("helo") + vector(hello))
}
private fun serverEnvelope(prefix: UByteArray): FlightEnvelope {
ensure(prefix.size >= 24, "Truncated server prefix")
val iv = prefix.copyOfRange(0, 24)
val root = derive(32, pair.publicKey, "server-reply", h1 + serverEncryptionPublic + trustedServer.keyBytes)
return FlightEnvelope(root, hash(label("server-flight") + h1 + iv), maximum - 24)
}
fun serverHelloSize(prefix: UByteArray): Int {
ensure(state == 0 && prefix.size == 44, "Unexpected server flight")
return 24 + serverEnvelope(prefix).size(prefix.copyOfRange(24, 44))
}
fun receiveServerHello(frame: UByteArray): UByteArray = guarded {
ensure(state == 0 && frame.size >= 44, "Unexpected server flight")
val (core, signature) = unpack(serverEnvelope(frame).open(frame.copyOfRange(24, frame.size)), 108, 64)
val reader = FlightReader(core)
validateRole(reader, carrier)
val serverDh = reader.take(32)
ensure(same(reader.take(32), hash(label("helo-decrypted") + pair.publicKey)), "Server proof mismatch")
ensure(same(reader.take(32), trustedServer.keyBytes), "Server identity mismatch")
peerBounds = PeerBounds.read(reader, carrier)
ensure(trustedServer.verify(signature, label("kiloparsec/concealed/1/server-auth") + h1 + vector(core)), "Server signature rejected")
h2 = hash(label("handshake") + vector(hello) + vector(frame))
keys = KeyExchange.clientSessionKeys(pair.publicKey, pair.secretKey, serverDh)
pair.secretKey.fill(0u)
val token = derive(32, keys.sendKey + keys.receiveKey, "connection-token", h2)
val clientKey = identity.verifyingKey.keyBytes
val signed = identity.sign(label("kiloparsec/concealed/1/client-auth") + token + h2 + clientKey)
ensure(signed.size == 64, "Suite requires Ed25519 signatures")
val ackCore = ubyteArrayOf(3u) + h2 + clientKey + signed
val iv = LibsodiumRandom.buf(24)
val root = derive(32, keys.sendKey, "finished/c2s", h2)
val sendMaximum = if (carrier == ConcealedCarrier.Kind.DATAGRAM) minOf(maximum, peerBounds.datagram) else maximum
val ack = iv + FlightEnvelope(root, hash(label("ack") + h2 + iv), sendMaximum - 24).seal(padded(ackCore, options, maximum = sendMaximum - 64))
root.fill(0u)
h3 = hash(label("authenticated-handshake") + h2 + vector(ack))
pendingSession = makeSession(keys, true, h2, h3, trustedServer, local, peerBounds)
state = 1
ack
}
private fun acceptanceEnvelope(prefix: UByteArray): FlightEnvelope {
val iv = prefix.copyOfRange(0, 24)
return FlightEnvelope(derive(32, keys.receiveKey, "finished/s2c", h2), hash(label("accept") + h2 + iv), maximum - 24)
}
fun acceptanceSize(prefix: UByteArray): Int {
ensure(state == 1 && prefix.size == 44, "Unexpected acceptance")
return 24 + acceptanceEnvelope(prefix).size(prefix.copyOfRange(24, 44))
}
fun receiveAcceptance(frame: UByteArray): HandshakeSession = guarded {
ensure(state == 1 && frame.size >= 44, "Unexpected acceptance")
val (core, _) = unpack(acceptanceEnvelope(frame).open(frame.copyOfRange(24, frame.size)), 33)
ensure(core[0] == 4u.toUByte() && same(core.copyOfRange(1, 33), h3), "Acceptance binding mismatch")
state = 2
keys.sendKey.fill(0u); keys.receiveKey.fill(0u)
pendingSession
}
/** Abandon on deadline, cancellation or authorization failure; never resume this attempt. */
fun destroy() {
if (state != 2 && ::pendingSession.isInitialized) {
pendingSession.token.fill(0u); pendingSession.sending.destroy(); pendingSession.receiving.destroy()
}
state = -1
pair.secretKey.fill(0u)
if (::keys.isInitialized) { keys.sendKey.fill(0u); keys.receiveKey.fill(0u) }
}
private inline fun <T> guarded(block: () -> T): T = try { block() } catch (failure: Exception) {
if (failure is UnauthenticatedFlight) throw failure
destroy()
throw if (failure is HandshakeFailure) failure else HandshakeFailure("Handshake cryptographic operation failed")
}
}
@@ -0,0 +1,143 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.generichash.GenericHash
import com.ionspin.kotlin.crypto.keyexchange.KeyExchangeSessionKeyPair
import com.ionspin.kotlin.crypto.secretbox.SecretBox
import com.ionspin.kotlin.crypto.secretbox.SecretBoxCorruptedOrTamperedDataExceptionOrInvalidKey
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import net.sergeych.crypto2.VerifyingPublicKey
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import net.sergeych.kiloparsec.concealed.record.RecordLimits
internal open class HandshakeFailure(message: String) : RuntimeException(message)
/** Ciphertext failed authentication; a UDP attempt may ignore it and wait for a valid flight. */
internal class UnauthenticatedFlight : HandshakeFailure("Handshake authentication failed")
internal data class HandshakeDirection(val streamKey: UByteArray, val streamSalt: UByteArray, val macKey: UByteArray, val udpKey: UByteArray) {
fun destroy() { streamKey.fill(0u); streamSalt.fill(0u); macKey.fill(0u); udpKey.fill(0u) }
}
internal data class HandshakeSession(
val token: UByteArray,
val peerIdentity: VerifyingPublicKey,
val sending: HandshakeDirection,
val receiving: HandshakeDirection,
val sendingLimits: RecordLimits,
val receivingLimits: RecordLimits,
)
internal data class PeerBounds(val body: Int, val datagram: Int, val padding: Int) {
fun bytes() = int32(body) + int16(datagram) + int32(padding)
companion object {
fun local(options: ConcealedOptions, kind: ConcealedCarrier.Kind) = PeerBounds(
options.limits.maxBodyBytes,
if (kind == ConcealedCarrier.Kind.DATAGRAM) options.udp.maxDatagramBytes else 0,
options.padding.maxBytes,
)
fun read(reader: FlightReader, kind: ConcealedCarrier.Kind): PeerBounds {
val value = PeerBounds(reader.int32(), reader.int16(), reader.int32())
ensure(value.body >= 256 && value.padding <= 1024 * 1024, "Invalid peer limits")
ensure(if (kind == ConcealedCarrier.Kind.STREAM) value.datagram == 0 else value.datagram in 512..65507, "Invalid datagram bound")
return value
}
}
}
internal fun text(s: String) = s.encodeToByteArray().asUByteArray()
internal fun int32(n: Int) = UByteArray(4) { (n ushr (24 - it * 8)).toUByte() }
internal fun int16(n: Int) = ubyteArrayOf((n ushr 8).toUByte(), n.toUByte())
internal fun vector(bytes: UByteArray) = int32(bytes.size) + bytes
internal fun label(s: String) = vector(text(s))
internal fun hash(bytes: UByteArray) = GenericHash.genericHash(bytes, 32)
internal fun derive(size: Int, secret: UByteArray, purpose: String, context: UByteArray) =
GenericHash.genericHash(label("kiloparsec/concealed/1") + label(purpose) + vector(context), size, secret)
internal fun ensure(condition: Boolean, message: String) { if (!condition) throw HandshakeFailure(message) }
internal fun same(a: UByteArray, b: UByteArray): Boolean {
if (a.size != b.size) return false
var different = 0
for (i in a.indices) different = different or (a[i].toInt() xor b[i].toInt())
return different == 0
}
internal fun carrierByte(kind: ConcealedCarrier.Kind) = if (kind == ConcealedCarrier.Kind.STREAM) 1u.toUByte() else 2u.toUByte()
internal fun validateRole(reader: FlightReader, kind: ConcealedCarrier.Kind) {
ensure(reader.take(1)[0] == 1u.toUByte(), "Unknown handshake suite")
ensure(reader.take(1)[0] == carrierByte(kind), "Wrong handshake carrier")
}
internal fun padded(core: UByteArray, options: ConcealedOptions, trailing: UByteArray = ubyteArrayOf(), maximum: Int = Int.MAX_VALUE): UByteArray {
val available = maximum - 4 - core.size - trailing.size
ensure(available >= 0, "Handshake core exceeds flight limit")
val size = LibsodiumRandom.uniform((minOf(options.padding.maxBytes, available) + 1).toUInt()).toInt()
return int32(core.size) + core + trailing + LibsodiumRandom.buf(size)
}
internal fun unpack(payload: UByteArray, coreSize: Int, tailSize: Int = 0): Pair<UByteArray, UByteArray> {
val reader = FlightReader(payload)
ensure(reader.int32() == coreSize, "Incorrect handshake core size")
return reader.take(coreSize) to reader.take(tailSize)
}
internal class FlightReader(private val bytes: UByteArray) {
private var cursor = 0
fun take(n: Int): UByteArray {
ensure(n >= 0 && n <= bytes.size - cursor, "Truncated handshake")
return bytes.copyOfRange(cursor, cursor + n).also { cursor += n }
}
fun int32(): Int {
val value = take(4).fold(0u) { n, b -> (n shl 8) or b.toUInt() }
ensure(value <= Int.MAX_VALUE.toUInt(), "Handshake length exceeds bound")
return value.toInt()
}
fun int16(): Int = take(2).fold(0) { n, b -> (n shl 8) or b.toInt() }
}
/** Header authentication precedes allocation; body repeats the exact authenticated length. */
internal class FlightEnvelope(private val root: UByteArray, private val context: UByteArray, private val maximum: Int) {
private fun key(name: String) = derive(32, root, "flight/$name-key", context)
private fun nonce(name: String) = derive(24, root, "flight/$name-iv", context)
fun seal(payload: UByteArray): UByteArray {
ensure(payload.size <= maximum - 40, "Handshake exceeds flight limit")
val length = int32(payload.size)
return SecretBox.easy(length, nonce("header"), key("header")) +
SecretBox.easy(length + payload, nonce("body"), key("body"))
}
fun size(header: UByteArray): Int {
ensure(header.size == 20, "Incorrect encrypted header size")
val length = open(header, "header")
val size = FlightReader(length).int32()
ensure(size <= maximum - 40, "Handshake exceeds flight limit")
return size + 40
}
fun open(frame: UByteArray): UByteArray {
ensure(frame.size >= 40 && frame.size <= maximum, "Incorrect handshake frame size")
val header = frame.copyOfRange(0, 20)
ensure(size(header) == frame.size, "Handshake frame size mismatch")
val body = open(frame.copyOfRange(20, frame.size), "body")
val length = FlightReader(open(header, "header")).int32()
ensure(same(body.copyOfRange(0, 4), int32(length)), "Handshake length binding mismatch")
return body.copyOfRange(4, body.size)
}
private fun open(bytes: UByteArray, part: String): UByteArray = try {
SecretBox.openEasy(bytes, nonce(part), key(part))
} catch (_: SecretBoxCorruptedOrTamperedDataExceptionOrInvalidKey) { throw UnauthenticatedFlight() }
}
internal fun flightLimit(options: ConcealedOptions, kind: ConcealedCarrier.Kind) =
if (kind == ConcealedCarrier.Kind.STREAM) options.limits.maxBootstrapBytes
else minOf(options.limits.maxBootstrapBytes, options.udp.maxDatagramBytes)
internal fun makeSession(
keys: KeyExchangeSessionKeyPair, client: Boolean, h2: UByteArray, h3: UByteArray,
peer: VerifyingPublicKey, local: PeerBounds, remote: PeerBounds,
): HandshakeSession {
val c2s = if (client) keys.sendKey else keys.receiveKey
val s2c = if (client) keys.receiveKey else keys.sendKey
fun direction(key: UByteArray, name: String) = HandshakeDirection(
derive(32, key, "traffic/stream/$name", h3), derive(24, key, "traffic/salt/$name", h3),
derive(32, key, "traffic/mac/$name", h3), derive(32, key, "traffic/udp/$name", h3),
)
val tx = if (client) "c2s" else "s2c"
val rx = if (client) "s2c" else "c2s"
fun limits(bounds: PeerBounds) = RecordLimits(bounds.body, minOf(bounds.padding, bounds.body - 4), if (bounds.datagram == 0) 1200 else bounds.datagram)
val outbound = PeerBounds(minOf(local.body, remote.body),
if (local.datagram == 0) 0 else minOf(local.datagram, remote.datagram), minOf(local.padding, remote.padding))
return HandshakeSession(derive(32, c2s + s2c, "connection-token", h2), peer,
direction(keys.sendKey, tx), direction(keys.receiveKey, rx), limits(outbound), limits(local))
}
@@ -0,0 +1,133 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.keyexchange.KeyExchange
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import net.sergeych.crypto2.SigningKey
import net.sergeych.crypto2.VerifyingPublicKey
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import net.sergeych.kiloparsec.concealed.elligator.ExperimentalElligator25519
/** The caller authorizes [verifyAck]'s identity BEFORE calling [accept]. */
internal class ServerHandshake(
serverSecret: UByteArray,
private val identity: SigningKey,
private val carrier: ConcealedCarrier.Kind,
private val options: ConcealedOptions,
heloFrame: UByteArray,
) {
val response: UByteArray
val clientDhPublicKey: UByteArray
private val maximum = flightLimit(options, carrier)
private val pair = KeyExchange.keypair()
private val local = PeerBounds.local(options, carrier)
private val peerBounds: PeerBounds
private val h2: UByteArray
private val keys: com.ionspin.kotlin.crypto.keyexchange.KeyExchangeSessionKeyPair
private var state = 0
private var h3: UByteArray = ubyteArrayOf()
private lateinit var verifiedSession: HandshakeSession
val session: HandshakeSession get() { check(state >= 1); return verifiedSession }
init {
require(serverSecret.size == 32 && identity.verifyingKey.keyBytes.size == 32)
ensure(heloFrame.size in 96..maximum, "Invalid HELO size")
val publicKey = ScalarMultiplication.scalarMultiplicationBase(serverSecret)
val (core, _) = unpack(helloEnvelope(heloFrame, serverSecret, maximum).open(heloFrame.copyOfRange(56, heloFrame.size)), 76)
val reader = FlightReader(core)
validateRole(reader, carrier)
clientDhPublicKey = reader.take(32)
ensure(same(reader.take(32), identity.verifyingKey.keyBytes), "Wrong intended identity")
peerBounds = PeerBounds.read(reader, carrier)
keys = KeyExchange.serverSessionKeys(pair.publicKey, pair.secretKey, clientDhPublicKey)
pair.secretKey.fill(0u)
val h1 = hash(label("helo") + vector(heloFrame))
val iv = LibsodiumRandom.buf(24)
val root = derive(32, clientDhPublicKey, "server-reply", h1 + publicKey + identity.verifyingKey.keyBytes)
val serverCore = ubyteArrayOf(1u, carrierByte(carrier)) + pair.publicKey +
hash(label("helo-decrypted") + clientDhPublicKey) + identity.verifyingKey.keyBytes + local.bytes()
val signature = identity.sign(label("kiloparsec/concealed/1/server-auth") + h1 + vector(serverCore))
ensure(signature.size == 64, "Suite requires Ed25519 signatures")
val sendMaximum = if (carrier == ConcealedCarrier.Kind.DATAGRAM) minOf(maximum, peerBounds.datagram, heloFrame.size * 3) else maximum
response = iv + FlightEnvelope(root, hash(label("server-flight") + h1 + iv), sendMaximum - 24).seal(padded(serverCore, options, signature, maximum = sendMaximum - 64))
root.fill(0u)
h2 = hash(label("handshake") + vector(heloFrame) + vector(response))
}
private fun ackEnvelope(prefix: UByteArray): FlightEnvelope {
val iv = prefix.copyOfRange(0, 24)
return FlightEnvelope(derive(32, keys.receiveKey, "finished/c2s", h2), hash(label("ack") + h2 + iv), maximum - 24)
}
fun ackSize(prefix: UByteArray): Int {
ensure(state == 0 && prefix.size == 44, "Unexpected client ACK")
return 24 + ackEnvelope(prefix).size(prefix.copyOfRange(24, 44))
}
fun verifyAck(frame: UByteArray): VerifyingPublicKey {
ensure(state == 0 && frame.size >= 44, "Unexpected client ACK")
// Unauthenticated UDP injection must not consume a pending attempt. Once envelope
// authentication succeeds, semantic/signature failure is terminal.
val authenticated = ackEnvelope(frame).open(frame.copyOfRange(24, frame.size))
state = -1
val (core, _) = unpack(authenticated, 129)
val reader = FlightReader(core)
ensure(reader.take(1)[0] == 3u.toUByte() && same(reader.take(32), h2), "ACK binding mismatch")
val client = VerifyingPublicKey(reader.take(32))
val signature = reader.take(64)
val token = derive(32, keys.receiveKey + keys.sendKey, "connection-token", h2)
ensure(client.verify(signature, label("kiloparsec/concealed/1/client-auth") + token + h2 + client.keyBytes), "Client signature rejected")
h3 = hash(label("authenticated-handshake") + h2 + vector(frame))
verifiedSession = makeSession(keys, false, h2, h3, client, local, peerBounds)
state = 1
return client
}
fun accept(): UByteArray {
ensure(state == 1, "Client must be verified and authorized first")
val iv = LibsodiumRandom.buf(24)
val root = derive(32, keys.sendKey, "finished/s2c", h2)
val core = ubyteArrayOf(4u) + h3
val sendMaximum = if (carrier == ConcealedCarrier.Kind.DATAGRAM) minOf(maximum, peerBounds.datagram) else maximum
val result = iv + FlightEnvelope(root, hash(label("accept") + h2 + iv), sendMaximum - 24).seal(padded(core, options, maximum = sendMaximum - 64))
root.fill(0u)
keys.sendKey.fill(0u); keys.receiveKey.fill(0u)
state = 2
return result
}
/** Call when abandoning a pending attempt; keep exact acceptance bytes for UDP retries externally. */
fun destroy() {
if (state != 2 && ::verifiedSession.isInitialized) {
verifiedSession.token.fill(0u); verifiedSession.sending.destroy(); verifiedSession.receiving.destroy()
}
state = -1
pair.secretKey.fill(0u)
keys.sendKey.fill(0u); keys.receiveKey.fill(0u)
}
companion object {
/** Same failed envelope stage only: never construct a handshake or create a response. */
internal fun discardHelloDecryption(frame: UByteArray, secret: UByteArray, maximum: Int, headerOnly: Boolean) {
val envelope = helloEnvelope(frame, secret, maximum)
if (headerOnly) envelope.size(frame.copyOfRange(56, 76))
else envelope.open(frame.copyOfRange(56, frame.size)).fill(0u)
}
fun helloSize(prefix: UByteArray, serverSecret: UByteArray, maximum: Int = 16 * 1024): Int {
ensure(prefix.size == 76, "Incorrect HELO prefix size")
return 56 + helloEnvelope(prefix, serverSecret, maximum).size(prefix.copyOfRange(56, 76))
}
private fun helloEnvelope(prefix: UByteArray, serverSecret: UByteArray, maximum: Int): FlightEnvelope {
require(serverSecret.size == 32)
ensure(prefix.size >= 56, "Truncated HELO prefix")
val publicKey = ScalarMultiplication.scalarMultiplicationBase(serverSecret)
val representative = prefix.copyOfRange(0, 32)
val context = prefix.copyOfRange(0, 56) + publicKey
val shared = ScalarMultiplication.scalarMultiplication(serverSecret, ExperimentalElligator25519.decode(representative))
ensure(shared.any { it != 0u.toUByte() }, "Rejected bootstrap DH")
val root = derive(32, shared, "bootstrap", context)
shared.fill(0u)
return FlightEnvelope(root, hash(context), maximum - 56)
}
}
}
@@ -0,0 +1,61 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.record
import com.ionspin.kotlin.crypto.generichash.GenericHash
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
/** Experimental wire limits. These do not disable authentication or change its tag size. */
internal data class RecordLimits(
val maxBodyBytes: Int = 16 * 1024 * 1024,
val maxPaddingBytes: Int = 255,
val maxDatagramBytes: Int = 1200,
val maxRecords: ULong = 1uL shl 32,
val maxStreamBytes: ULong = 1uL shl 40,
) {
init {
require(maxBodyBytes in 4..(Int.MAX_VALUE - 64))
require(maxPaddingBytes in 0..maxBodyBytes - 4)
require(maxDatagramBytes in 52..65507)
require(maxRecords in 1uL..(1uL shl 32))
require(maxStreamBytes in 1uL..(1uL shl 40))
}
}
internal open class InvalidRecord(message: String) : RuntimeException(message)
/** Unauthenticated UDP input is discarded without disturbing an established session. */
internal class UnauthenticatedDatagram(message: String) : InvalidRecord(message)
internal fun u32(value: Int): UByteArray = UByteArray(4) { (value ushr (24 - it * 8)).toUByte() }
internal fun readU32(bytes: UByteArray, offset: Int = 0): Int {
val value = (0..3).fold(0u) { v, i -> (v shl 8) or bytes[offset + i].toUInt() }
if (value > Int.MAX_VALUE.toUInt()) throw InvalidRecord("Length exceeds implementation bound")
return value.toInt()
}
internal fun u64(value: ULong): UByteArray = UByteArray(8) { (value shr (56 - it * 8)).toUByte() }
internal fun readU64(bytes: UByteArray): ULong = (0..7).fold(0uL) { v, i -> (v shl 8) or bytes[i].toULong() }
internal fun paddedBody(objectBytes: UByteArray, limits: RecordLimits, cap: Int = limits.maxBodyBytes): UByteArray {
val maximum = minOf(limits.maxBodyBytes, cap)
require(objectBytes.size <= maximum - 4) { "Object exceeds protected record budget" }
val paddingLimit = minOf(limits.maxPaddingBytes, maximum - 4 - objectBytes.size)
val paddingLength = LibsodiumRandom.uniform((paddingLimit + 1).toUInt()).toInt()
return u32(objectBytes.size) + objectBytes + LibsodiumRandom.buf(paddingLength)
}
/** Call only after the complete body has been authenticated. */
internal fun unpackBody(body: UByteArray): UByteArray {
if (body.size < 4) throw InvalidRecord("Missing object length")
val size = readU32(body)
if (size > body.size - 4) throw InvalidRecord("Object exceeds authenticated body")
return body.copyOfRange(4, 4 + size)
}
internal fun recordMac(key: UByteArray, sequence: ULong, length: UByteArray, body: UByteArray): UByteArray {
val state = GenericHash.genericHashInit(16, key)
GenericHash.genericHashUpdate(state, "kiloparsec/concealed/record-mac/v1".encodeToByteArray().asUByteArray())
GenericHash.genericHashUpdate(state, u64(sequence))
GenericHash.genericHashUpdate(state, length)
GenericHash.genericHashUpdate(state, body)
return GenericHash.genericHashFinal(state)
}
@@ -0,0 +1,204 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.record
import com.ionspin.kotlin.crypto.generichash.GenericHash
import com.ionspin.kotlin.crypto.stream.Stream
import com.ionspin.kotlin.crypto.util.LibsodiumUtil
/**
* Continuous ChaCha20, retaining partial 64-byte blocks across arbitrary input boundaries.
* The 24-byte handshake nonce is domain-separated into a fresh key and 8-byte nonce, since
* the existing multiplatform binding does not expose XChaCha20 consistently.
* Experimental: never construct a second stream with the same key and initial nonce.
*/
internal class ContinuousCipher(key: UByteArray, nonce: UByteArray) {
private val streamKey: UByteArray
private val streamNonce: UByteArray
private var block = 0uL
private var exhausted = false
private var mask = ubyteArrayOf()
private var maskOffset = 0
private var destroyed = false
init {
require(key.size == 32 && nonce.size == 24)
streamKey = GenericHash.genericHash(
"kiloparsec/concealed/stream-key/v1".encodeToByteArray().asUByteArray() + nonce, 32, key)
streamNonce = GenericHash.genericHash(
"kiloparsec/concealed/stream-nonce/v1".encodeToByteArray().asUByteArray() + nonce, 16, key).copyOf(8)
}
fun apply(input: UByteArray): UByteArray {
check(!destroyed) { "Stream cipher destroyed" }
val output = UByteArray(input.size)
var offset = 0
if (maskOffset < mask.size) {
val n = minOf(input.size, mask.size - maskOffset)
repeat(n) { output[it] = (input[it].toInt() xor mask[maskOffset + it].toInt()).toUByte() }
maskOffset += n
offset += n
}
// Bulk calls avoid one native round trip per 64 bytes for large objects.
val full = (input.size - offset) / 64 * 64
if (full > 0) {
val blocks = full.toULong() / 64u
reserve(blocks)
val encrypted = Stream.chacha20XorIc(input.copyOfRange(offset, offset + full), streamNonce, block, streamKey)
encrypted.copyInto(output, offset)
advance(blocks)
offset += full
}
if (offset < input.size) {
reserve(1uL)
if (mask.isNotEmpty()) LibsodiumUtil.memzero(mask)
mask = Stream.chacha20XorIc(UByteArray(64), streamNonce, block, streamKey)
advance(1uL)
val n = input.size - offset
repeat(n) { output[offset + it] = (input[offset + it].toInt() xor mask[it].toInt()).toUByte() }
maskOffset = n
}
return output
}
private fun reserve(count: ULong) {
check(!exhausted && count - 1uL <= ULong.MAX_VALUE - block) { "Stream counter exhausted" }
}
private fun advance(count: ULong) {
if (count - 1uL == ULong.MAX_VALUE - block) exhausted = true else block += count
}
/** Invoke only after writers/readers have stopped; lifecycle access is externally serialized. */
fun destroy() {
destroyed = true
exhausted = true
LibsodiumUtil.memzero(streamKey)
LibsodiumUtil.memzero(streamNonce)
if (mask.isNotEmpty()) LibsodiumUtil.memzero(mask)
mask = ubyteArrayOf()
maskOffset = 0
block = 0uL
}
}
/** Single writer. Encode only after priority selection, in actual wire order. */
internal class TcpRecordEncoder(
key: UByteArray, macKey: UByteArray, initialNonce: UByteArray,
private val limits: RecordLimits = RecordLimits(),
) {
private val cipher = ContinuousCipher(key, initialNonce)
private val macKey = macKey.copyOf().also { require(it.size == 32) }
private var sequence = 0uL
private var exhausted = false
private var encryptedBytes = 0uL
fun encode(objectBytes: UByteArray): UByteArray {
check(!exhausted) { "Record sequence exhausted" }
check(sequence < limits.maxRecords) { "Stream record usage ceiling reached" }
val body = paddedBody(objectBytes, limits)
val length = u32(body.size)
val size = body.size.toULong() + 20uL
check(size <= limits.maxStreamBytes - encryptedBytes) { "Stream byte usage ceiling reached" }
val wire = cipher.apply(length + body + recordMac(macKey, sequence, length, body))
encryptedBytes += size
if (sequence == ULong.MAX_VALUE) exhausted = true else sequence++
return wire
}
fun destroy() {
exhausted = true
cipher.destroy()
LibsodiumUtil.memzero(macKey)
sequence = 0uL
encryptedBytes = 0uL
}
}
/** Incremental bounded receiver. No inner object bytes escape before MAC verification. */
internal class TcpRecordDecoder(
key: UByteArray, macKey: UByteArray, initialNonce: UByteArray,
private val limits: RecordLimits = RecordLimits(),
) {
private val cipher = ContinuousCipher(key, initialNonce)
private val macKey = macKey.copyOf().also { require(it.size == 32) }
private val header = UByteArray(4)
private var headerUsed = 0
private var bodyAndTag = ubyteArrayOf()
private var bodyUsed = 0
private var sequence = 0uL
private var exhausted = false
private var failed = false
private var decryptedBytes = 0uL
val hasPartialRecord: Boolean get() = headerUsed != 0
fun feed(ciphertext: UByteArray): List<UByteArray> {
check(!failed) { "Record decoder already failed" }
val output = mutableListOf<UByteArray>()
var offset = 0
try {
while (offset < ciphertext.size) {
if (exhausted) throw InvalidRecord("Record sequence exhausted")
if (sequence >= limits.maxRecords) throw InvalidRecord("Stream record usage ceiling reached")
if (headerUsed < 4) {
val n = minOf(4 - headerUsed, ciphertext.size - offset)
if (n.toULong() > limits.maxStreamBytes - decryptedBytes)
throw InvalidRecord("Stream byte usage ceiling reached")
cipher.apply(ciphertext.copyOfRange(offset, offset + n)).copyInto(header, headerUsed)
decryptedBytes += n.toULong()
headerUsed += n
offset += n
if (headerUsed < 4) continue
val length = readU32(header)
if (length !in 4..limits.maxBodyBytes) throw InvalidRecord("Protected body exceeds bounds")
if (length.toULong() + 16uL > limits.maxStreamBytes - decryptedBytes)
throw InvalidRecord("Stream byte usage ceiling reached")
bodyAndTag = UByteArray(length + 16)
bodyUsed = 0
}
val n = minOf(bodyAndTag.size - bodyUsed, ciphertext.size - offset)
cipher.apply(ciphertext.copyOfRange(offset, offset + n)).copyInto(bodyAndTag, bodyUsed)
bodyUsed += n
decryptedBytes += n.toULong()
offset += n
if (bodyUsed == bodyAndTag.size) {
val body = bodyAndTag.copyOfRange(0, bodyAndTag.size - 16)
val tag = bodyAndTag.copyOfRange(body.size, bodyAndTag.size)
if (!LibsodiumUtil.memcmp(tag, recordMac(macKey, sequence, header, body)))
throw InvalidRecord("Stream record authentication failed")
output += unpackBody(body)
if (sequence == ULong.MAX_VALUE) exhausted = true else sequence++
headerUsed = 0
bodyAndTag = ubyteArrayOf()
bodyUsed = 0
}
}
return output
} catch (e: Exception) {
failed = true
throw e
}
}
/** Transport must independently require an authenticated CLOSE object for clean EOF. */
fun finish() {
check(!failed) { "Record decoder already failed" }
if (hasPartialRecord) {
failed = true
throw InvalidRecord("Truncated stream record")
}
}
fun destroy() {
failed = true
exhausted = true
cipher.destroy()
LibsodiumUtil.memzero(macKey)
LibsodiumUtil.memzero(header)
if (bodyAndTag.isNotEmpty()) LibsodiumUtil.memzero(bodyAndTag)
bodyAndTag = ubyteArrayOf()
headerUsed = 0
bodyUsed = 0
sequence = 0uL
decryptedBytes = 0uL
}
}
@@ -0,0 +1,69 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.record
import com.ionspin.kotlin.crypto.secretbox.SecretBox
import com.ionspin.kotlin.crypto.secretbox.SecretBoxCorruptedOrTamperedDataExceptionOrInvalidKey
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import com.ionspin.kotlin.crypto.util.LibsodiumUtil
import net.sergeych.kiloparsec.ReplayWindow
internal data class EncodedDatagram(val sequence: ULong, val bytes: UByteArray)
/** One direction, externally serialized together with queue admission. Never reuse session keys. */
internal class UdpRecordEncoder(key: UByteArray, private val limits: RecordLimits = RecordLimits()) {
private val key = key.copyOf().also { require(it.size == 32) }
private var sequence = 0uL
private var exhausted = false
fun encode(objectBytes: UByteArray): EncodedDatagram {
check(!exhausted) { "Record sequence exhausted; reconnect with fresh keys" }
check(sequence < limits.maxRecords) { "Datagram usage ceiling reached; reconnect" }
val body = paddedBody(objectBytes, limits, limits.maxDatagramBytes - 48)
val nonce = LibsodiumRandom.buf(24)
val current = sequence
val ciphertext = SecretBox.easy(u64(current) + body, nonce, key)
if (current == ULong.MAX_VALUE) exhausted = true else sequence++
return EncodedDatagram(current, nonce + ciphertext)
}
fun destroy() {
exhausted = true
LibsodiumUtil.memzero(key)
sequence = 0uL
}
}
/** One receive direction. Authentication is checked before any replay-window state changes. */
internal class UdpRecordDecoder(key: UByteArray, private val limits: RecordLimits = RecordLimits()) {
private val key = key.copyOf().also { require(it.size == 32) }
private val replay = ReplayWindow()
private var acceptedRecords = 0uL
private var destroyed = false
/** Null means authenticated duplicate/stale packet; malformed packets are failures. */
fun decode(datagram: UByteArray): UByteArray? {
check(!destroyed) { "Datagram decoder destroyed" }
if (datagram.size !in 52..limits.maxDatagramBytes) throw UnauthenticatedDatagram("Invalid datagram size")
val plaintext = try {
SecretBox.openEasy(datagram.copyOfRange(24, datagram.size), datagram.copyOfRange(0, 24), key)
} catch (_: SecretBoxCorruptedOrTamperedDataExceptionOrInvalidKey) {
throw UnauthenticatedDatagram("Datagram authentication failed")
}
if (plaintext.size - 8 > limits.maxBodyBytes) throw InvalidRecord("Datagram body exceeds limit")
val sequence = readU64(plaintext)
if (sequence >= limits.maxRecords) throw InvalidRecord("Datagram sequence exceeds usage ceiling")
if (replay.containsOrStale(sequence)) return null
if (acceptedRecords >= limits.maxRecords) throw InvalidRecord("Datagram usage ceiling reached")
val result = unpackBody(plaintext.copyOfRange(8, plaintext.size))
replay.record(sequence)
acceptedRecords++
return result
}
fun destroy() {
destroyed = true
LibsodiumUtil.memzero(key)
acceptedRecords = 0uL
}
}
@@ -0,0 +1,174 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.*
import net.sergeych.kiloparsec.*
import kotlin.test.*
import kotlin.time.Duration.Companion.seconds
class ConcealedClientTest {
private class Credentials {
val encryption = DecryptingSecretKey.new()
val server = SigningSecretKey.new()
val client = SigningSecretKey.new()
}
private val options = ConcealedOptions(
handshake = ConcealedOptions.Handshake(timeoutMillis = 15_000, maxAttempts = 3, retryDelayMillis = 0),
padding = ConcealedOptions.Padding(0),
)
private class MemoryCarrier(
private val input: Channel<UByteArray>,
private val output: Channel<UByteArray>,
) : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.STREAM
var closed = false
override suspend fun receive() = input.receiveCatching().getOrNull()
override suspend fun send(bytes: UByteArray) { output.send(bytes.copyOf()) }
override suspend fun close() { closed = true; input.close(); output.close() }
}
private fun carriers(): Pair<MemoryCarrier, MemoryCarrier> {
val a = Channel<UByteArray>(16)
val b = Channel<UByteArray>(16)
return MemoryCarrier(a, b) to MemoryCarrier(b, a)
}
private fun client(credentials: Credentials, local: KiloInterface<Unit> = KiloInterface(),
options: ConcealedOptions = this.options, connect: suspend () -> ConcealedCarrier) =
ConcealedClient(local, Unit, credentials.encryption.publicKey, credentials.server.verifyingKey,
credentials.client, options, connect)
@Test fun finiteConnectorFailuresReleaseEarlyRpc() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
var attempts = 0
val failure = IllegalStateException("Connector failed")
val client = client(credentials) { attempts++; throw failure }
val echo by command<Int, Int>()
val early = async { runCatching { client.call(echo, 1) } }
val result = runCatching { client.run() }
// Coroutine stack-trace recovery may copy ordinary JVM exception instances.
val runFailure = result.exceptionOrNull()
val queuedFailure = early.await().exceptionOrNull()
assertIs<IllegalStateException>(runFailure, runFailure?.stackTraceToString())
assertIs<IllegalStateException>(queuedFailure, queuedFailure?.stackTraceToString())
assertEquals(failure.message, runFailure.message)
assertEquals(failure.message, queuedFailure.message)
assertEquals(3, attempts)
client.close()
}
@Test fun timedOutAttemptsUseFreshCarriersAndFreshHellos() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val hellos = mutableListOf<UByteArray>()
var closed = 0
var attempts = 0
val client = client(credentials, options = options.copy(handshake = options.handshake.copy(timeoutMillis = 100))) {
attempts++
object : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.STREAM
private var isClosed = false
override suspend fun receive(): UByteArray? = awaitCancellation()
override suspend fun send(bytes: UByteArray) { hellos += bytes.copyOf() }
override suspend fun close() { if (!isClosed) { isClosed = true; closed++ } }
}
}
assertTrue(runCatching { client.run() }.isFailure)
assertEquals(3, attempts)
assertEquals(3, closed)
assertEquals(3, hellos.size)
for (i in hellos.indices) for (j in 0 until i) {
assertFalse(hellos[i].contentEquals(hellos[j]))
assertFalse(hellos[i].copyOfRange(0, 32).contentEquals(hellos[j].copyOfRange(0, 32)))
}
client.close()
}
@Test fun authenticatedPolicyAndSignatureFailuresAreNotRetried() = runTest(timeout = 60.seconds) {
initCrypto()
for (badSignature in listOf(false, true)) {
val credentials = Credentials()
val jobs = mutableListOf<Job>()
var attempts = 0
val policyFailure = LocalInterface.BreakConnectionException("Reject authenticated peer")
val local = KiloInterface<Unit>().apply {
if (!badSignature) onConnected { throw policyFailure }
}
val signer = if (!badSignature) credentials.server else object : SigningKey by credentials.server {
override fun sign(message: UByteArray) = credentials.server.sign(message).also {
it[0] = (it[0].toInt() xor 1).toUByte()
}
}
val client = client(credentials, local) {
attempts++
val (a, b) = carriers()
val server = ConcealedConnection.server(KiloInterface<Unit>(), b, Unit,
ConcealedServerIdentity(credentials.encryption, signer), options)
jobs += launch { runCatching { server.run() } }
a
}
try {
val failure = runCatching { client.run() }.exceptionOrNull()
assertNotNull(failure)
if (!badSignature) assertSame(policyFailure, failure)
assertEquals(1, attempts)
} finally { client.close(); jobs.forEach { it.cancelAndJoin() } }
}
}
@Test fun closeDuringBlockedConnectorCancelsRunAndReleasesWaiters() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val entered = CompletableDeferred<Unit>()
val exited = CompletableDeferred<Unit>()
val client = client(credentials) {
entered.complete(Unit)
try { awaitCancellation() } finally { exited.complete(Unit) }
}
val echo by command<Int, Int>()
val early = async { runCatching { client.call(echo, 1) } }
val running = async { runCatching { client.run() } }
entered.await()
client.close()
exited.await()
assertTrue(running.await().isFailure)
assertTrue(early.await().isFailure)
}
@Test fun reconnectBeforeEstablishmentDoesNotReplayUncertainApplicationCall() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val commit by command<Int, Int>()
val enteredCommand = CompletableDeferred<Unit>()
var executions = 0
var attempts = 0
lateinit var server: ConcealedConnection<Unit>
var serverRun: Job? = null
val client = client(credentials) {
attempts++
if (attempts == 1) throw IllegalStateException("First endpoint unavailable")
val (a, b) = carriers()
val local = KiloInterface<Unit>().apply {
on(commit) { executions++; enteredCommand.complete(Unit); awaitCancellation() }
}
server = ConcealedConnection.server(local, b, Unit,
ConcealedServerIdentity(credentials.encryption, credentials.server), options)
serverRun = launch { runCatching { server.run() } }
a
}
val running = async { runCatching { client.run() } }
val call = async { runCatching { client.call(commit, 9) } }
try {
client.awaitReady()
enteredCommand.await()
server.close()
assertTrue(call.await().isFailure)
running.await()
assertEquals(2, attempts)
assertEquals(1, executions)
} finally { client.close(); serverRun?.cancelAndJoin(); running.cancelAndJoin(); call.cancelAndJoin() }
}
}
@@ -0,0 +1,263 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.*
import kotlin.test.*
import kotlin.time.Duration.Companion.seconds
class ConcealedConnectionTest {
private class MemoryCarrier(
override val kind: ConcealedCarrier.Kind,
private val incoming: Channel<UByteArray>,
private val outgoing: Channel<UByteArray>,
) : ConcealedCarrier {
var sent = 0
var dropFlight = -1
var duplicate = false
var noiseBeforeFlights: Set<Int> = emptySet()
var trickleData = false
var trickleBytesSent = 0
override suspend fun send(bytes: UByteArray) {
sent++
if (sent == dropFlight) return
if (kind == ConcealedCarrier.Kind.DATAGRAM) {
if (sent in noiseBeforeFlights) {
// Short junk cannot supply a header; the second packet keeps an authentic
// header but corrupts the separately authenticated body.
outgoing.send(ubyteArrayOf(9u, 8u, 7u))
outgoing.send(bytes.copyOf().also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() })
}
outgoing.send(bytes.copyOf())
if (duplicate) outgoing.send(bytes.copyOf())
} else {
if (trickleData) {
// Twenty fragments at 100 ms intervals outlast a 1 s whole-record
// deadline, while each individual read remains within that deadline.
repeat(minOf(20, bytes.size)) { i ->
if (i != 0) delay(100)
outgoing.send(bytes.copyOfRange(i, i + 1))
trickleBytesSent++
}
awaitCancellation()
}
var position = 0
var step = 0
val sizes = intArrayOf(1, 3, 7, 64, 2, 127)
while (position < bytes.size) {
val end = minOf(bytes.size, position + sizes[step++ % sizes.size])
outgoing.send(bytes.copyOfRange(position, end))
position = end
}
}
}
override suspend fun receive(): UByteArray? = incoming.receiveCatching().getOrNull()
override suspend fun close() { incoming.close(); outgoing.close() }
}
private fun carriers(kind: ConcealedCarrier.Kind): Pair<MemoryCarrier, MemoryCarrier> {
val a = Channel<UByteArray>(64)
val b = Channel<UByteArray>(64)
return MemoryCarrier(kind, a, b) to MemoryCarrier(kind, b, a)
}
private class Credentials {
val serverEncryption = DecryptingSecretKey.new()
val serverSigning = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val serverIdentity = ConcealedServerIdentity(serverEncryption, serverSigning)
}
private val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
@Test fun segmentedStreamSupportsDeferredInitializationDuplexPushAndErrors() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.STREAM)
val echo by command<Int, Int>()
val notify by command<Int, Unit>()
val fail by command<Int, Int>()
val notifications = Channel<Int>(16)
val initializedCall = CompletableDeferred<Int>()
val clientLocal = KiloInterface<Unit>().apply {
on(echo) { it + 100 }
on(notify) { notifications.send(it) }
onConnected {
assertEquals(credentials.serverSigning.verifyingKey, remoteIdentity)
val r = remote
launch { initializedCall.complete(r.call(echo, 12)) }
}
}
val serverLocal = KiloInterface<Unit>().apply {
on(echo) { it * 2 }
on(notify) { notifications.send(it) }
on(fail) { throw IllegalArgumentException("Expected remote failure") }
onConnected { assertEquals(credentials.clientSigning.verifyingKey, remoteIdentity) }
}
val client = ConcealedConnection.client(clientLocal, a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(serverLocal, b, Unit, credentials.serverIdentity, options)
val clientState = CompletableDeferred<Unit>()
val clientRun = async { runCatching { client.run { if (it) clientState.complete(Unit) } } }
val serverRun = async { runCatching { server.run() } }
val early = async { client.call(echo, 7) }
try {
clientState.await(); server.awaitReady()
assertEquals(14, early.await())
assertEquals(24, initializedCall.await())
assertContentEquals(client.token(), server.token())
assertEquals(credentials.clientSigning.verifyingKey, server.remoteId())
coroutineScope {
val requests = (0 until 8).map { i -> async { assertEquals(i * 2, client.call(echo, i)) } } +
(0 until 8).map { i -> async { assertEquals(i + 100, server.call(echo, i)) } }
requests.awaitAll()
}
client.push(notify, 31); server.push(notify, 47)
assertEquals(setOf(31, 47), setOf(notifications.receive(), notifications.receive()))
assertFailsWith<RemoteInterface.RemoteException> { client.call(fail, 1) }
assertEquals(18, client.call(echo, 9))
client.close()
withTimeout(2_000) { clientRun.await(); serverRun.await() }
} finally { client.close(); server.close(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin() }
}
@Test fun udpLostAckAndAcceptanceRecoverAndDuplicatesDoNotRepeatRpc() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.DATAGRAM)
// HELO/send1, ACK/send2; first client ACK and server acceptance are lost.
a.dropFlight = 2
b.dropFlight = 2
val echo by command<Int, Int>()
var executions = 0
val serverLocal = KiloInterface<Unit>().apply { on(echo) { executions++; it + 1 } }
val client = ConcealedConnection.client(KiloInterface<Unit>(), a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(serverLocal, b, Unit, credentials.serverIdentity, options)
val clientRun = async { runCatching { client.run() } }
val serverRun = async { runCatching { server.run() } }
try {
client.awaitReady(); server.awaitReady()
assertTrue(a.sent >= 4, "Lost ACK and lost acceptance each require an ACK retry")
a.duplicate = true; b.duplicate = true
assertEquals(43, client.call(echo, 42))
assertEquals(8, client.call(echo, 7))
assertEquals(2, executions)
assertTrue(clientRun.isActive); assertTrue(serverRun.isActive)
} finally { client.close(); server.close(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin() }
}
@Test fun udpIgnoresUnauthenticatedNoiseAcrossAllReplyFlights() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.DATAGRAM)
a.noiseBeforeFlights = setOf(2) // ACK
b.noiseBeforeFlights = setOf(1, 2) // server hello and acceptance
val echo by command<Int, Int>()
var executions = 0
val client = ConcealedConnection.client(KiloInterface<Unit>(), a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(KiloInterface<Unit>().apply { on(echo) { executions++; it } },
b, Unit, credentials.serverIdentity, options)
val clientRun = async { runCatching { client.run() } }
val serverRun = async { runCatching { server.run() } }
try {
client.awaitReady(); server.awaitReady()
assertEquals(51, client.call(echo, 51))
assertEquals(1, executions)
assertTrue(clientRun.isActive); assertTrue(serverRun.isActive)
assertNull(client.terminalFailure); assertNull(server.terminalFailure)
} finally { client.close(); server.close(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin() }
}
@Test fun negotiatedStreamLimitRejectsOversizeLocallyAndKeepsConnectionUsable() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.STREAM)
val echo by command<String, String>()
var executed = 0
val client = ConcealedConnection.client(KiloInterface<Unit>(), a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(KiloInterface<Unit>().apply { on(echo) { executed++; it } },
b, Unit, credentials.serverIdentity,
options.copy(limits = ConcealedOptions.Limits(maxBodyBytes = 256)))
val clientRun = async { runCatching { client.run() } }
val serverRun = async { runCatching { server.run() } }
try {
client.awaitReady(); server.awaitReady()
assertFailsWith<IllegalArgumentException> { client.call(echo, "x".repeat(1024)) }
assertEquals("small", client.call(echo, "small"))
assertEquals(1, executed)
assertTrue(clientRun.isActive); assertTrue(serverRun.isActive)
} finally { client.close(); server.close(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin() }
}
@Test fun slowTrickleDoesNotRestartAbsolutePartialRecordDeadline() = runTest(timeout = 60.seconds) {
initCrypto()
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.STREAM)
val echo by command<Int, Int>()
var executed = 0
val client = ConcealedConnection.client(KiloInterface<Unit>(), a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(KiloInterface<Unit>().apply { on(echo) { executed++; it } },
b, Unit, credentials.serverIdentity,
options.copy(limits = ConcealedOptions.Limits(partialRecordTimeoutMillis = 1_000)))
// Connection jobs use real timers because production measures Monotonic elapsed time;
// virtual test-scheduler time cannot advance that clock. Count progress, not a tight
// wall-clock duration, so the test distinguishes a reset-on-every-read implementation.
val clientRun = async(Dispatchers.Default) { runCatching { client.run() } }
val serverRun = async(Dispatchers.Default) { runCatching { server.run() } }
var call: Deferred<Result<Int>>? = null
try {
client.awaitReady(); server.awaitReady()
a.trickleData = true
val pendingCall = async(Dispatchers.Default) { runCatching { client.call(echo, 1) } }
call = pendingCall
withContext(Dispatchers.Default) {
withTimeout(5_000) {
serverRun.await()
clientRun.await()
assertTrue(pendingCall.await().isFailure)
}
}
assertEquals(0, executed, "Unauthenticated partial objects must not reach dispatch")
assertTrue(a.trickleBytesSent in 1 until 20,
"Whole-record deadline must expire while fragments continue arriving; sent ${a.trickleBytesSent}")
} finally {
client.close(); server.close()
call?.cancelAndJoin(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin()
}
}
@Test fun policyRejectionOnEitherEndFailsQueuedCallsWithoutDispatch() = runTest(timeout = 60.seconds) {
initCrypto()
for (rejectServer in listOf(true, false)) {
val credentials = Credentials()
val (a, b) = carriers(ConcealedCarrier.Kind.STREAM)
val echo by command<Int, Int>()
var executed = 0
var announced = false
val rejection = LocalInterface.BreakConnectionException("Policy rejected")
val clientLocal = KiloInterface<Unit>()
val serverLocal = KiloInterface<Unit>().apply { on(echo) { executed++; it } }
(if (rejectServer) serverLocal else clientLocal).onConnected { throw rejection }
val client = ConcealedConnection.client(clientLocal, a, Unit, credentials.serverEncryption.publicKey,
credentials.serverSigning.verifyingKey, credentials.clientSigning, options)
val server = ConcealedConnection.server(serverLocal, b, Unit, credentials.serverIdentity, options)
val queued = async { runCatching { client.call(echo, 5) } }
val clientRun = async { runCatching { client.run { if (it) announced = true } } }
val serverRun = async { runCatching { server.run() } }
try {
assertTrue(queued.await().isFailure)
withTimeout(2_000) { clientRun.await(); serverRun.await() }
assertFalse(announced)
assertEquals(0, executed)
if (!rejectServer) assertSame(rejection, clientRun.await().exceptionOrNull())
} finally { client.close(); server.close(); clientRun.cancelAndJoin(); serverRun.cancelAndJoin() }
}
}
}
@@ -0,0 +1,125 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.CancellationException
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.async
import kotlinx.coroutines.awaitAll
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.concealed.handshake.*
import kotlin.test.*
class HelloFailureNoiseTest {
private val seed get() = UByteArray(32) { (it + 11).toUByte() }
private fun noise() = HelloFailureNoise(seed) { UByteArray(it) { 255u } }
@Test fun onlyEnvelopeAuthenticationFailureTriggersExactlyOneDiscardedAttempt() {
val noise = noise()
var calls = 0
assertEquals(42, noise.run({ 42 }) { calls++ })
assertEquals(0, calls)
val malformed = HandshakeFailure("Invalid length or low-order DH")
assertSame(malformed, assertFailsWith<HandshakeFailure> { noise.run({ throw malformed }) { calls++ } })
assertEquals(0, calls)
val original = UnauthenticatedFlight()
assertSame(original, assertFailsWith<UnauthenticatedFlight> { noise.run({ throw original }) { calls++ } })
assertEquals(1, calls) // A successful dummy result never becomes a real accepted result.
}
@Test fun mutatesOnlyEffectiveDistinctBitsAndErasesTemporaryCopy() {
val originalSeed = seed
val snapshot = originalSeed.copyOf()
val noise = HelloFailureNoise(originalSeed) { UByteArray(it) { 255u } }
var retained: UByteArray? = null
var mutated: UByteArray? = null
assertFailsWith<UnauthenticatedFlight> {
noise.run({ throw UnauthenticatedFlight() }) { retained = it; mutated = it.copyOf() }
}
val changed = (0..255).filter { bit ->
((snapshot[bit / 8].toInt() xor mutated!![bit / 8].toInt()) and (1 shl (bit % 8))) != 0
}
assertEquals(8, changed.size)
assertTrue(changed.all { it in 3..253 })
assertContentEquals(snapshot, originalSeed)
assertTrue(retained!!.all { it == 0u.toUByte() })
}
@Test fun expectedDummyFailurePreservesOriginalButCancellationAndRuntimeErrorsPropagate() {
val noise = noise()
val original = UnauthenticatedFlight()
assertSame(original, assertFailsWith<UnauthenticatedFlight> {
noise.run({ throw original }) { throw HandshakeFailure("Dummy rejected") }
})
val cancelled = CancellationException("Cancelled")
assertSame(cancelled, assertFailsWith<CancellationException> {
noise.run({ throw original }) { throw cancelled }
})
val bug = IllegalStateException("Unexpected failure")
assertSame(bug, assertFailsWith<IllegalStateException> {
noise.run({ throw original }) { throw bug }
})
val fatal = object : Error("Fatal") {}
assertSame(fatal, assertFailsWith<Error> { noise.run({ throw original }) { throw fatal } })
}
@Test fun concurrentFailuresKeepSeedImmutableAndCopiesIndependent() = runTest {
val noise = noise()
val mutations = (0 until 24).map {
async(Dispatchers.Default) {
var mutation: UByteArray? = null
var retained: UByteArray? = null
assertFailsWith<UnauthenticatedFlight> {
noise.run({ throw UnauthenticatedFlight() }) { key ->
mutation = key.copyOf(); retained = key
}
}
assertTrue(retained!!.all { it == 0u.toUByte() })
mutation!!
}
}.awaitAll()
mutations.forEach { assertContentEquals(mutations.first(), it) }
assertFalse(mutations.first().contentEquals(seed))
}
@Test fun actualHeaderAndBodyFailuresUseDecoyWithoutMutatingRealKeyOrAccepting() = runTest {
initCrypto()
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val identity = ConcealedServerIdentity(encryption, signing)
val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
val client = ClientHandshake(encryption.publicKey.keyBytes, signing.verifyingKey,
SigningSecretKey.new(), ConcealedCarrier.Kind.STREAM, options)
val hello = client.hello
val realKey = encryption.keyBytes.copyOf()
val maximum = options.limits.maxBootstrapBytes
val prefix = hello.copyOfRange(0, 76)
assertEquals(hello.size, identity.helloAuthentication(prefix, maximum, true, true) {
ServerHandshake.helloSize(prefix, encryption.keyBytes, maximum)
})
for (enabled in listOf(false, true)) {
val badHeader = prefix.copyOf().also { it[75] = (it[75].toInt() xor 1).toUByte() }
assertFailsWith<UnauthenticatedFlight> {
identity.helloAuthentication(badHeader, maximum, true, enabled) {
ServerHandshake.helloSize(badHeader, encryption.keyBytes, maximum)
}
}
val badBody = hello.copyOf().also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() }
assertFailsWith<UnauthenticatedFlight> {
identity.helloAuthentication(badBody, maximum, false, enabled) {
ServerHandshake(encryption.keyBytes, signing, ConcealedCarrier.Kind.STREAM, options, badBody)
}
}
}
assertContentEquals(realKey, encryption.keyBytes)
// Original unmodified HELO still establishes the usual authenticated server response.
val server = identity.helloAuthentication(hello, maximum, false, true) {
ServerHandshake(encryption.keyBytes, signing, ConcealedCarrier.Kind.STREAM, options, hello)
}
try { assertTrue(client.receiveServerHello(server.response).isNotEmpty()) }
finally { client.destroy(); server.destroy() }
}
}
@@ -0,0 +1,84 @@
package net.sergeych.kiloparsec.concealed.elligator
// Independent reference vectors, retrieved 2026-10-22 from https://elligator.org/vectors/.
// Direct vectors retain r and u (v is not used by X25519); failed inverse vectors use an empty result.
internal object ElligatorReferenceVectors {
val direct = listOf(
Pair("0000000000000000000000000000000000000000000000000000000000000000", "0000000000000000000000000000000000000000000000000000000000000000"),
Pair("66665895c5bc6e44ba8d65fd9307092e3244bf2c18877832bd568cb3a2d38a12", "04d44290d13100b2c25290c9343d70c12ed4813487a07ac1176daa5925e7975e"),
Pair("673a505e107189ee54ca93310ac42e4545e9e59050aaac6f8b5f64295c8ec02f", "242ae39ef158ed60f20b89396d7d7eef5374aba15dc312a6aea6d1e57cacf85e"),
Pair("990b30e04e1c3620b4162b91a33429bddb9f1b70f1da6e5f76385ed3f98ab131", "998e98021eb4ee653effaa992f3fae4b834de777a953271baaa1fa3fef6b776e"),
Pair("341a60725b482dd0de2e25a585b208433044bc0a1ba762442df3a0e888ca063c", "683a71d7fca4fc6ad3d4690108be808c2e50a5af3174486741d0a83af52aeb01"),
Pair("922688fa428d42bc1fa8806998fbc5959ae801817e85a42a45e8ec25a0d7541a", "696f341266c64bcfa7afa834f8c34b2730be11c932e08474d1a22f26ed82410b"),
Pair("0d3b0eb88b74ed13d5f6a130e03c4ad607817057dc227152827c0506a538bb3a", "0b00df174d9fb0b6ee584d2cf05613130bad18875268c38b377e86dfefef177f"),
Pair("01a3ea5658f4e00622eeacf724e0bd82068992fae66ed2b04a8599be16662e35", "7ae4c58bc647b5646c9f5ae4c2554ccbf7c6e428e7b242a574a5a9c293c21f7e"),
Pair("1d991dff82a84afe97874c0f03a60a56616a15212fbe10d6c099aa3afcfabe35", "f81f235696f81df90ac2fc861ceee517bff611a394b5be5faaee45584642fb0a"),
Pair("185435d2b005a3b63f3187e64a1ef3582533e1958d30e4e4747b4d1d3376c728", "f938b1b320abb0635930bd5d7ced45ae97fa8b5f71cc21d87b4c60905c125d34"),
Pair("b98d026c2a247fd113596f4317d8ff9f89f172c4be46320a5e82f579b953b633", "19f98122afbd38dd5b97afc07fe89c00d9d45834a25ae851e527e34a5e763c07"),
Pair("69599ab5a829c3e9515128d368da7354a8b69fcee4e34d0a668b783b6cae550f", "09024abaaef243e3b69366397e8dfc1fdc14a0ecc7cf497cbe4f328839acce69"),
Pair("409f76202dcc3280d14a9bc87b5f474349bbb6325e92733e0feee571a7f2462a", "16235355f04bef29019ec6903788b9ee56214d1fdb60ddcc9f9bdc2816a3b132"),
Pair("9172922f96d2fa41ea0daf961857056f1656ab8406db80eaeae76af58f8c9f10", "beab745a2a4b4e7f1a7335c3ffcdbd85139f3a72b667a01ee3e3ae0e530b3372"),
Pair("6850a20ac5b6d2fa7af7042ad5be234d3311b9fb303753dd2b610bd566983201", "1287388eb2beeff706edb9cf4fcfdd35757f22541b61528570b86e8915be1530"),
Pair("84417826c0e80af7cb25a73af1ba87594ff7048a26248b5757e52f2824e06831", "51acd2e8910e7d28b4993db7e97e2b995005f26736f60dcdde94bdf8cb542251"),
)
val inverse = listOf(
Triple("0000000000000000000000000000000000000000000000000000000000000000", "0", "0000000000000000000000000000000000000000000000000000000000000000"),
Triple("01a3ea5658f4e00622eeacf724e0bd82068992fae66ed2b04a8599be16662e35", "0", ""),
Triple("01a3ea5658f4e00622eeacf724e0bd82068992fae66ed2b04a8599be16662e35", "1", ""),
Triple("f569bb6b6ac92fecbd0ed9f37055c08c7eb49b3325eb415d4cc5d3c44a27d854", "0", ""),
Triple("f569bb6b6ac92fecbd0ed9f37055c08c7eb49b3325eb415d4cc5d3c44a27d854", "1", ""),
Triple("2af3d4b274cb8f1eafe97a8e78155026b0e97d14844fb17e67463a2a159a3d72", "0", ""),
Triple("2af3d4b274cb8f1eafe97a8e78155026b0e97d14844fb17e67463a2a159a3d72", "1", ""),
Triple("94268b1888c4a743c5a25281a68701c01fabf8676002ed1079255bcad7b42f60", "0", ""),
Triple("94268b1888c4a743c5a25281a68701c01fabf8676002ed1079255bcad7b42f60", "1", ""),
Triple("808afb062244775fe768995402c10b2dcffb0f65b8ba872dc659902da89d2b49", "0", ""),
Triple("808afb062244775fe768995402c10b2dcffb0f65b8ba872dc659902da89d2b49", "1", ""),
Triple("727b81b1455f79c59077c6abeb4b38e4b98b63cce8e5270f1b2d8b42789b7b63", "0", ""),
Triple("727b81b1455f79c59077c6abeb4b38e4b98b63cce8e5270f1b2d8b42789b7b63", "1", ""),
Triple("97350b0977d06146bd7609bfadc8af713d3c97af2929625e44ccdb66791c5427", "0", ""),
Triple("97350b0977d06146bd7609bfadc8af713d3c97af2929625e44ccdb66791c5427", "1", ""),
Triple("f85393caffdd45ff11884d57a11d636497d2c051bfddd1dd2daa720a3ad0e41e", "0", ""),
Triple("f85393caffdd45ff11884d57a11d636497d2c051bfddd1dd2daa720a3ad0e41e", "1", ""),
Triple("0d281e20ec2e3561a20ebc908c1c5e9ffb1f850e597ffc91c8861d1c57ef3a3e", "0", ""),
Triple("0d281e20ec2e3561a20ebc908c1c5e9ffb1f850e597ffc91c8861d1c57ef3a3e", "1", ""),
Triple("c68d4ceead9a188d00ddcda391c5c036ecd5cd281f243aacaa327bbbf3384c3a", "0", ""),
Triple("c68d4ceead9a188d00ddcda391c5c036ecd5cd281f243aacaa327bbbf3384c3a", "1", ""),
Triple("120d71dd9fc64bd6e2fcf8e96a9d22817f2dc31b314ba484f87cb6073af5a541", "0", ""),
Triple("120d71dd9fc64bd6e2fcf8e96a9d22817f2dc31b314ba484f87cb6073af5a541", "1", ""),
Triple("621744f808a1b0eb252c85628e06214319260092f1d17ce1c509e1a250308063", "0", ""),
Triple("621744f808a1b0eb252c85628e06214319260092f1d17ce1c509e1a250308063", "1", ""),
Triple("918348cc15950e2d21940da1f6f26e0111ac3a84af976fbb85d872b8d3375d23", "0", ""),
Triple("918348cc15950e2d21940da1f6f26e0111ac3a84af976fbb85d872b8d3375d23", "1", ""),
Triple("ca4e9d3319c955caf99827a0dce25d4d3d21330740c024bab3a1cf65dda8fa2e", "0", ""),
Triple("ca4e9d3319c955caf99827a0dce25d4d3d21330740c024bab3a1cf65dda8fa2e", "1", ""),
Triple("230eca35f689c16da6bcb0641863eaebcf05a5aeba5abf94c88ccb66f7953e65", "0", ""),
Triple("230eca35f689c16da6bcb0641863eaebcf05a5aeba5abf94c88ccb66f7953e65", "1", ""),
Triple("7a6685fd00c9ac478ae4878017ef4f4816dc8489f5fbb8122af687a8c7fc173d", "0", ""),
Triple("7a6685fd00c9ac478ae4878017ef4f4816dc8489f5fbb8122af687a8c7fc173d", "1", ""),
Triple("fd6fa7f8d40e82d9cdcbb2b3d137887b639db5eaeb1e20a6651e9e288c6b1d73", "0", "bdf0babcc57bfce100e81ddc84f92d5021c4065e2457a4adaa2916bffa16ca16"),
Triple("fd6fa7f8d40e82d9cdcbb2b3d137887b639db5eaeb1e20a6651e9e288c6b1d73", "1", "b0986f7934908cec837208e9cb94182b1ea5cd24875c02c409e9892ea48ecf0b"),
Triple("230030447df3d9a9634091503eb94ee39737a8690c3f61bb122c7264bc00083d", "0", "905c2d8787d2c9ab775aac1ab7e0c51e5363e0d7b5915a899dfff5d889b5c410"),
Triple("230030447df3d9a9634091503eb94ee39737a8690c3f61bb122c7264bc00083d", "1", "161db55c3b12ac9087d3c3c94e598896dc9ed792900172aac450b41d5177ab3f"),
Triple("d298c05fd81f21dd1c27a2b0d75089d50ed25a91d42a7cd3a4b86de6f5f43a26", "0", "d1f6409bda55a0851c23085de23054af757fb89955da242b1a1e96c27180653d"),
Triple("d298c05fd81f21dd1c27a2b0d75089d50ed25a91d42a7cd3a4b86de6f5f43a26", "1", "ce9e93b505f84992fe84a0b171bc9b4e4ec876221b807811f343784478d76b19"),
Triple("9b9e4dc144e41a5e0ff2123b6601d26828b301fa70afc778c49fb5399d44f04e", "0", "c16580d060e3856894abf4c7a166e0812ab79e6df00e53662182febd37901704"),
)
val scalar = listOf(
Pair("8099a76a3a4391bb45729a026cf8f6d1cdbb40d3e77887cd42a9734cba58eada", "d39fbbfc405a199815f9fe0b9746e5486197058b4cfafde734be3194686d8c0f"),
Pair("49e9d46e5ccbd6422460e48f0e2591a089c7a12c0cea9c9cb9e59f8da4b7d22f", "d053808acf4da6f20220b6b863e321b66ec0c43e35b41a359fdcf35cd8059653"),
Pair("9ae801817e85a42a45e8ec2541afa9340d3b0eb88b74ed13d5f6a130e03c4ad6", "609ba8ccc80db0270cc27ab1e45e36c316f72864e9ae2e8b8642ec0f714f4f48"),
Pair("ab29c3e9515128d368da7354a8b69fcee4e34d0a668b783bd85cab1ead6089df", "cda87a924c2ddc705f6b7a7149437d635357912f9e78c66fc10d68720d56c131"),
Pair("1c57056f1656ab8406db80eaeae76af51f193f216850a20ac5b6d2fa7af7042a", "85f6bfaa7cebf45110460a49f4780483b7dd30fac0c0a1c0a66731386e369d38"),
Pair("25248b5757e52f2849c0d162b0fbe152849f49034d2fa00ccc7b960fad7b30b6", "9f122d2a60ff9e6f0535c795c1f4b49896ff3790e479ee3a81bdbbdfe5406c25"),
Pair("161bc00f4e8f8903225948a02d00e67dc70f65536dffa0c516cc2e9cecd94ff7", "70c74173fc142fab7964ae7f7745a19941f5866efef44a65ce9db90f14e3e61b"),
Pair("6fc92fecbd0ed9f37055c08c7eb49b3325eb415d4cc5d3c4954eb0a92af3d4b2", "f48c70397977d0497db6a5743824264ee1b7297bc277b34b6c1d52d08c0fa124"),
Pair("a8e97a8e78155026b0e97d14844fb17e67463a2a2b347be4a0ca9ff75afae655", "aacd264dc444edcf628ea4292761934eabcd758a1fedcb23bcf3d9076ad5b355"),
Pair("711c0eefb61268cc99aac1d994268b1888c4a743c5a25281a68701c01fabf867", "68a6ae56f0c632089760076681c98bd9f4feb6f6510507aa8b22f40e1cc9c501"),
Pair("92560e9f24733c921b06af336543c8bf370c9122ba24ed87e7a2dae487a2b53b", "012969335b29c682b6de162ba7e86154512c7aeeb828b776bb2fc6e3d3578d5a"),
Pair("cb7164b5fabb82a297350b0977d06146bd7609bfadc8af713d3c97af2929625e", "e44c139d516b3874c289f83d513b33d267c8d03d9c0990ff19fd9ca4e84ced53"),
Pair("f438a84e4de3fc72e672419c49d5ab43c350d54f1b85babc2efe147ebed9fb7a", "634fa5f71b3573833ec9721c435b9f8558cc933acd4afeb309387e75fbd31518"),
Pair("758ad64ce74eb3b940dab7502492a6d8813466639de65213b5fee99d5f12044d", "ebe1131ab10943be9938973314ca29e37f65102a908c8d76907af64c72e2a234"),
Pair("564295833f191208f85393caffdd45ff11884d57a11d636497d2c051bfddd1dd", "e1dae044dada772b295c2fab6826c78acfba49f6956b48b165f91d8e1888cb31"),
Pair("a770e05be3fb8b289b524cfa730bc2bd3a6ebb487036bb097e2235db4f6b17df", "9f135038101b6d809fbd4fb047362049189089d5ace38a0243175d5b5aabde1b"),
)
}
@@ -0,0 +1,105 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.elligator
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import kotlin.math.ceil
import kotlin.math.sqrt
import kotlin.random.Random
import kotlin.time.TimeSource
/** Opt-in diagnostic, not a statistical proof of constant-time execution. No key bytes are emitted. */
internal object ElligatorTimingProbe {
private var sink = 0
private data class Sample(val group: String, val nanos: Long, val attempts: Int = 0)
fun run(count: Int, warmup: Int, metadata: String): String {
require(count in 16..100_000 && warmup in 0..10_000)
val rng = Random(237404)
val samples = mutableListOf<Sample>()
val publicRandom = List(count.coerceAtLeast(warmup).coerceAtLeast(16)) {
UByteArray(32) { rng.nextInt(256).toUByte() }
}
fun measured(group: String, record: Boolean, action: () -> Unit) {
val start = TimeSource.Monotonic.markNow()
action()
val ns = start.elapsedNow().inWholeNanoseconds
if (record) samples += Sample(group, ns)
}
fun consume(value: UByteArray?) { if (value != null) sink = sink xor value[0].toInt() }
val valid = ElligatorReferenceVectors.inverse.filter { it.third.isNotEmpty() && !it.first.all { c -> c == '0' } }
val invalid = ElligatorReferenceVectors.inverse.filter { it.third.isEmpty() }
fun hex(s: String) = s.chunked(2).map { it.toInt(16).toUByte() }.toUByteArray()
val validInputs = valid.map { hex(it.first) to it.second.toInt() }
val invalidInputs = invalid.map { hex(it.first) to it.second.toInt() }
// Interleave randomized class order each round to reduce monotonic JIT/thermal drift.
repeat(warmup + count) { round ->
val record = round >= warmup
val index = round % publicRandom.size
val secret = publicRandom[index].copyOf()
val torsionInputs = (0..7).map { low -> secret.copyOf().also { it[0] = ((it[0].toInt() and 248) or low).toUByte() } }
val randomRep = publicRandom[index]
val paddedRep = randomRep.copyOf().also { it[31] = (it[31].toInt() or 192).toUByte() }
val jobs = mutableListOf<Pair<String, () -> Unit>>()
jobs += "decode.zero" to { consume(ExperimentalElligator25519.decode(UByteArray(32))) }
jobs += "decode.random" to { consume(ExperimentalElligator25519.decode(randomRep)) }
jobs += "decode.same-random-high-bits" to { consume(ExperimentalElligator25519.decode(paddedRep)) }
for (low in 0..7) jobs += "special.torsion-$low" to { consume(ExperimentalElligator25519.specialPublicKey(torsionInputs[low])) }
val good = validInputs[round % validInputs.size]
val bad = invalidInputs[round % invalidInputs.size]
jobs += "encode.representable" to { consume(ExperimentalElligator25519.encode(good.first, good.second)) }
jobs += "encode.unrepresentable" to { consume(ExperimentalElligator25519.encode(bad.first, bad.second)) }
jobs += "generate.actual" to {
val pair = ExperimentalElligator25519.generateKeyPair()
consume(pair.representative)
pair.destroy()
}
for ((name, job) in jobs.shuffled(rng)) measured(name, record, job)
torsionInputs.forEach { it.fill(0u) }
secret.fill(0u)
}
// Instrumented equivalent loop exposes rejection counts without changing production code.
repeat(count) {
var attempts = 0
val start = TimeSource.Monotonic.markNow()
while (true) {
attempts++
val secret = LibsodiumRandom.buf(32)
val encoded = try {
ExperimentalElligator25519.encode(ExperimentalElligator25519.specialPublicKey(secret), LibsodiumRandom.buf(1)[0].toInt())
} finally { secret.fill(0u) }
if (encoded != null) { consume(encoded); break }
}
samples += Sample("generate.instrumented", start.elapsedNow().inWholeNanoseconds, attempts)
}
publicRandom.forEach { it.fill(0u) }
val groups = samples.groupBy { it.group }
fun mean(values: List<Sample>) = values.map { it.nanos.toDouble() }.average()
fun variance(values: List<Sample>): Double {
val m = mean(values)
return values.sumOf { (it.nanos - m) * (it.nanos - m) } / (values.size - 1)
}
fun welch(a: List<Sample>, b: List<Sample>): Double =
(mean(a) - mean(b)) / sqrt(variance(a) / a.size + variance(b) / b.size)
return buildString {
appendLine("Elligator timing diagnostic; $metadata")
appendLine("seed=237404 samples-per-class=$count warmup-rounds=$warmup clock=TimeSource.Monotonic units=ns sink=$sink")
appendLine("Classes: decode inputs are PUBLIC (zero is pathological); paired high-bit inputs decode identically. special classes share the same clamped scalar each round and differ only in low 3 secret torsion bits. encode classes intentionally compare observable success/failure control flow. generate includes fresh synthetic randomness and rejection tails.")
appendLine("Interpretation: exploratory Welch t only, no p-value/constant-time claim; unequal inputs, autocorrelation, GC/JIT, scheduler noise and multiple comparisons remain. Synthetic keys only; key bytes omitted.")
appendLine("group,n,mean_ns,median_ns,p95_ns,p99_ns,max_ns")
groups.forEach { (name, data) ->
val ordered = data.map { it.nanos }.sorted()
fun q(f: Double) = ordered[(ceil(f * ordered.size).toInt() - 1).coerceIn(0, ordered.lastIndex)]
appendLine("$name,${data.size},${mean(data)},${q(0.5)},${q(0.95)},${q(0.99)},${ordered.last()}")
}
appendLine("welch_group_a,welch_group_b,t")
for (i in 1..7) appendLine("special.torsion-0,special.torsion-$i,${welch(groups.getValue("special.torsion-0"), groups.getValue("special.torsion-$i"))}")
appendLine("encode.representable,encode.unrepresentable,${welch(groups.getValue("encode.representable"), groups.getValue("encode.unrepresentable"))}")
appendLine("decode.random,decode.same-random-high-bits,${welch(groups.getValue("decode.random"), groups.getValue("decode.same-random-high-bits"))}")
appendLine("attempts,count")
groups.getValue("generate.instrumented").groupingBy { it.attempts }.eachCount().entries.sortedBy { it.key }.forEach { (a, n) -> appendLine("$a,$n") }
appendLine("raw_group,nanos,attempts")
samples.forEach { appendLine("${it.group},${it.nanos},${it.attempts}") }
}
}
}
@@ -0,0 +1,84 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.elligator
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.initCrypto
import kotlin.test.Test
import kotlin.test.assertContentEquals
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
import kotlin.test.assertNotNull
import kotlin.test.assertNull
import kotlin.time.Duration.Companion.seconds
class ExperimentalElligator25519Test {
@Test fun independentDirectVectors() {
ElligatorReferenceVectors.direct.forEach { (representative, expected) ->
val bytes = hex(representative)
for (padding in listOf(0, 64, 128, 192)) {
bytes[31] = ((bytes[31].toInt() and 63) or padding).toUByte()
assertContentEquals(hex(expected), ExperimentalElligator25519.decode(bytes))
}
}
}
@Test fun independentInverseVectors() {
ElligatorReferenceVectors.inverse.forEach { (publicKey, sign, expected) ->
val encoded = ExperimentalElligator25519.encode(hex(publicKey), sign.toInt())
if (expected.isEmpty()) assertNull(encoded) else {
assertContentEquals(hex(expected), encoded)
assertContentEquals(hex(publicKey), ExperimentalElligator25519.decode(assertNotNull(encoded)))
}
}
}
@Test fun independentFullCurveMultiplicationVectors() = runTest(timeout = 60.seconds) {
initCrypto()
ElligatorReferenceVectors.scalar.forEach { (secret, expected) ->
assertContentEquals(hex(expected), ExperimentalElligator25519.specialPublicKey(hex(secret)))
}
}
@Test fun agreementWithUnmodifiedLibsodiumAndAllTorsionComponents() = runTest(timeout = 60.seconds) {
initCrypto()
val serverSecret = LibsodiumRandom.buf(32)
val serverPublic = ScalarMultiplication.scalarMultiplicationBase(serverSecret)
val secret = LibsodiumRandom.buf(32)
val normalPublic = ScalarMultiplication.scalarMultiplicationBase(secret)
val expected = ScalarMultiplication.scalarMultiplication(serverSecret, normalPublic)
for (lowBits in 0..7) {
secret[0] = ((secret[0].toInt() and 248) or lowBits).toUByte()
val fullPublic = ExperimentalElligator25519.specialPublicKey(secret)
assertContentEquals(expected, ScalarMultiplication.scalarMultiplication(serverSecret, fullPublic))
assertContentEquals(expected, ScalarMultiplication.scalarMultiplication(secret, serverPublic))
}
repeat(8) {
val pair = ExperimentalElligator25519.generateKeyPair()
val decoded = ExperimentalElligator25519.decode(pair.representative)
assertContentEquals(
ScalarMultiplication.scalarMultiplication(serverSecret, decoded),
ScalarMultiplication.scalarMultiplication(pair.secretKey, serverPublic)
)
pair.destroy()
assertEquals(true, pair.secretKey.all { it == 0.toUByte() })
}
}
@Test fun lengthsAndLowOrderRepresentative() = runTest {
initCrypto()
for (length in listOf(0, 31, 33)) {
assertFailsWith<IllegalArgumentException> { ExperimentalElligator25519.decode(UByteArray(length)) }
assertFailsWith<IllegalArgumentException> { ExperimentalElligator25519.encode(UByteArray(length), 0) }
assertFailsWith<IllegalArgumentException> { ExperimentalElligator25519.specialPublicKey(UByteArray(length)) }
}
val zero = ExperimentalElligator25519.decode(UByteArray(32))
assertContentEquals(UByteArray(32), zero)
// Mapping is total; agreement must reject low-order points, not silently derive a key.
kotlin.test.assertFails { ScalarMultiplication.scalarMultiplication(LibsodiumRandom.buf(32), zero) }
}
private fun hex(value: String) = value.chunked(2).map { it.toInt(16).toUByte() }.toUByteArray()
}
@@ -0,0 +1,150 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.SigningKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import kotlin.test.*
import kotlin.time.Duration.Companion.seconds
class ConcealedHandshakeTest {
private val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
private class Credentials {
val secret = LibsodiumRandom.buf(32)
val public = ScalarMultiplication.scalarMultiplicationBase(secret)
val server = SigningSecretKey.new()
val client = SigningSecretKey.new()
}
private fun Credentials.attempt(kind: ConcealedCarrier.Kind = ConcealedCarrier.Kind.STREAM): Pair<ClientHandshake, ServerHandshake> {
val c = ClientHandshake(public, server.verifyingKey, client, kind, options)
val s = ServerHandshake(secret, server, kind, options, c.hello)
return c to s
}
@Test fun allFourFlightsKeysTokenAndPeerIdentities() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
for (kind in ConcealedCarrier.Kind.entries) {
val (client, server) = creds.attempt(kind)
assertEquals(176, client.hello.size)
assertEquals(client.hello.size, ServerHandshake.helloSize(client.hello.copyOfRange(0, 76), creds.secret))
assertEquals(240, server.response.size)
assertEquals(server.response.size, client.serverHelloSize(server.response.copyOfRange(0, 44)))
assertFails { server.accept() }
val ack = client.receiveServerHello(server.response)
assertEquals(197, ack.size)
assertEquals(ack.size, server.ackSize(ack.copyOfRange(0, 44)))
assertEquals(creds.client.verifyingKey, server.verifyAck(ack))
val acceptance = server.accept()
assertEquals(101, acceptance.size)
assertEquals(acceptance.size, client.acceptanceSize(acceptance.copyOfRange(0, 44)))
val session = client.receiveAcceptance(acceptance)
assertEquals(creds.server.verifyingKey, session.peerIdentity)
assertEquals(creds.client.verifyingKey, server.session.peerIdentity)
assertContentEquals(session.token, server.session.token)
assertContentEquals(session.sending.streamKey, server.session.receiving.streamKey)
assertContentEquals(session.sending.streamSalt, server.session.receiving.streamSalt)
assertContentEquals(session.sending.macKey, server.session.receiving.macKey)
assertContentEquals(session.sending.udpKey, server.session.receiving.udpKey)
assertContentEquals(session.receiving.udpKey, server.session.sending.udpKey)
assertFalse(session.receiving.udpKey.contentEquals(session.sending.udpKey))
assertFails { server.verifyAck(ack) }
assertFails { client.receiveAcceptance(acceptance) }
}
}
@Test fun eachAttemptIsFreshAndOldAckCannotAuthenticateNewAttempt() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
val (c1, s1) = creds.attempt()
val ack1 = c1.receiveServerHello(s1.response)
s1.verifyAck(ack1)
val session1 = c1.receiveAcceptance(s1.accept())
val (c2, s2) = creds.attempt()
assertFalse(c1.hello.contentEquals(c2.hello))
val ack2 = c2.receiveServerHello(s2.response)
assertFailsWith<UnauthenticatedFlight> { s2.verifyAck(ack1) }
// Unauthenticated injection/replay cannot consume a valid pending UDP attempt.
s2.verifyAck(ack2)
c2.receiveAcceptance(s2.accept())
val (c3, s3) = creds.attempt()
s3.verifyAck(c3.receiveServerHello(s3.response))
val session3 = c3.receiveAcceptance(s3.accept())
assertFalse(session1.token.contentEquals(session3.token))
assertFalse(session1.sending.udpKey.contentEquals(session3.sending.udpKey))
}
@Test fun alteredRepresentativeAliasAndWrongStaticServerKeyFail() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
val (c, _) = creds.attempt()
val alias = c.hello.copyOf()
alias[31] = (alias[31].toInt() xor 128).toUByte()
assertFails { ServerHandshake(creds.secret, creds.server, ConcealedCarrier.Kind.STREAM, options, alias) }
assertFails { ServerHandshake(LibsodiumRandom.buf(32), creds.server, ConcealedCarrier.Kind.STREAM, options, c.hello) }
assertFails { ServerHandshake(creds.secret, SigningSecretKey.new(), ConcealedCarrier.Kind.STREAM, options, c.hello) }
assertFails { ServerHandshake(creds.secret, creds.server, ConcealedCarrier.Kind.DATAGRAM, options, c.hello) }
}
@Test fun corruptServerSignatureAndCorruptClientSignatureAreRejected() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
val badServer = corruptSigner(creds.server)
val client = ClientHandshake(creds.public, creds.server.verifyingKey, creds.client, ConcealedCarrier.Kind.STREAM, options)
val server = ServerHandshake(creds.secret, badServer, ConcealedCarrier.Kind.STREAM, options, client.hello)
assertFails { client.receiveServerHello(server.response) }
val badClient = ClientHandshake(creds.public, creds.server.verifyingKey, corruptSigner(creds.client), ConcealedCarrier.Kind.STREAM, options)
val goodServer = ServerHandshake(creds.secret, creds.server, ConcealedCarrier.Kind.STREAM, options, badClient.hello)
assertFails { goodServer.verifyAck(badClient.receiveServerHello(goodServer.response)) }
assertFails { goodServer.accept() }
}
@Test fun mutationOfEachProtectedFlightFails() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
fun corrupt(frame: UByteArray) = frame.copyOf().also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() }
val (c1, s1) = creds.attempt()
assertFailsWith<UnauthenticatedFlight> { c1.receiveServerHello(corrupt(s1.response)) }
s1.verifyAck(c1.receiveServerHello(s1.response))
c1.receiveAcceptance(s1.accept())
val (c2, s2) = creds.attempt()
val ack2 = c2.receiveServerHello(s2.response)
assertFailsWith<UnauthenticatedFlight> { s2.verifyAck(corrupt(ack2)) }
s2.verifyAck(ack2)
c2.receiveAcceptance(s2.accept())
val (c3, s3) = creds.attempt()
s3.verifyAck(c3.receiveServerHello(s3.response))
val acceptance = s3.accept()
assertFailsWith<UnauthenticatedFlight> { c3.receiveAcceptance(corrupt(acceptance)) }
c3.receiveAcceptance(acceptance)
}
@Test fun largePaddingPolicyShrinksToDatagramBudget() = runTest(timeout = 60.seconds) {
initCrypto()
val creds = Credentials()
val clientOptions = ConcealedOptions(padding = ConcealedOptions.Padding(10000))
val serverOptions = ConcealedOptions(padding = ConcealedOptions.Padding(10000), udp = ConcealedOptions.Udp(maxDatagramBytes = 512))
val kind = ConcealedCarrier.Kind.DATAGRAM
val fittingClient = ClientHandshake(creds.public, creds.server.verifyingKey, creds.client, kind,
clientOptions.copy(udp = ConcealedOptions.Udp(maxDatagramBytes = 512)))
val server = ServerHandshake(creds.secret, creds.server, kind, serverOptions, fittingClient.hello)
assertTrue(fittingClient.hello.size <= 512)
assertTrue(server.response.size <= 512)
val ack = fittingClient.receiveServerHello(server.response)
assertTrue(ack.size <= 512)
server.verifyAck(ack)
val accept = server.accept()
assertTrue(accept.size <= 512)
fittingClient.receiveAcceptance(accept)
}
private fun corruptSigner(key: SigningKey): SigningKey = object : SigningKey by key {
override fun sign(message: UByteArray) = key.sign(message).also { it[0] = (it[0].toInt() xor 1).toUByte() }
}
}
@@ -0,0 +1,43 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.secretbox.SecretBox
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.initCrypto
import kotlin.test.*
class FlightEnvelopeTest {
@Test fun independentKdfVectors() = runTest {
initCrypto()
val secret = UByteArray(32) { it.toUByte() }
val context = ubyteArrayOf(0xa0u, 0xa1u, 0xa2u, 0xa3u)
assertContentEquals(hex("595859a9e444bb24d1df7c118eeef70fea86720249bd9292eba6e32744da6f19"), derive(32, secret, "vector", context))
assertContentEquals(hex("46d4b4de3827562b5654864913c2c46cb9802a6a71322599"), derive(24, secret, "vector", context))
}
@Test fun headerAuthenticationLengthBoundsAndRepeatedLengthBinding() = runTest {
initCrypto()
val root = UByteArray(32) { it.toUByte() }
val context = UByteArray(32) { (it + 40).toUByte() }
val envelope = FlightEnvelope(root, context, 128)
val payload = ubyteArrayOf(10u, 20u, 30u)
val frame = envelope.seal(payload)
assertEquals(43, frame.size)
assertEquals(frame.size, envelope.size(frame.copyOfRange(0, 20)))
assertContentEquals(payload, envelope.open(frame))
val modifiedHeader = frame.copyOfRange(0, 20).also { it[0] = (it[0].toInt() xor 1).toUByte() }
assertFailsWith<HandshakeFailure> { envelope.size(modifiedHeader) }
assertFailsWith<HandshakeFailure> { envelope.open(frame.copyOf(frame.size - 1)) }
assertFailsWith<HandshakeFailure> { envelope.open(frame + ubyteArrayOf(0u)) }
fun encryptHeader(length: Int) = SecretBox.easy(int32(length),
derive(24, root, "flight/header-iv", context), derive(32, root, "flight/header-key", context))
// Correct authentication does not allow the peer to force an oversized allocation.
assertFailsWith<HandshakeFailure> { envelope.size(encryptHeader(89)) }
val differentBody = SecretBox.easy(int32(4) + payload,
derive(24, root, "flight/body-iv", context), derive(32, root, "flight/body-key", context))
assertFailsWith<HandshakeFailure> { envelope.open(encryptHeader(3) + differentBody) }
}
private fun hex(value: String) = value.chunked(2).map { it.toInt(16).toUByte() }.toUByteArray()
}
@@ -0,0 +1,61 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.handshake
// Generated by tools/concealed-vectors/reference.py; synthetic secrets ONLY.
internal object IndependentWireFixtures {
fun bytes(name: String): UByteArray = values.getValue(name).chunked(2).map { it.toUByte(16) }.toUByteArray()
private val values = mapOf(
"serverSecret" to "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f",
"serverPublic" to "8f40c5adb68f25624ae5b214ea767a6ec94d829d3d7b5e1ad1ba6f3e2138285f",
"clientDhSecret" to "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f",
"clientDhPublic" to "358072d6365880d1aeea329adf9121383851ed21a28e3b75e965d0d2cd166254",
"serverDhSecret" to "404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f",
"serverDhPublic" to "79a631eede1bf9c98f12032cdeadd0e7a079398fc786b88cc846ec89af85a51a",
"serverSigningSeed" to "606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f",
"serverIdentity" to "174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd5",
"clientSigningSeed" to "808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f",
"clientIdentity" to "cd14b37f956e953194ff7fb73b3d81dcc561d61a7538094b7c3e1a643ee5f3aa",
"representative" to "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf",
"heloIv" to "c0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7",
"bootstrapShared" to "900702f8b7a7dd869cf98f17dce31515a39e8043636352e8bf7be99043b83101",
"bootstrapRoot" to "f0cce9735754e6985895bbf2ad73a16a1ad5732399df991a598f17e71f3b2d04",
"heloCore" to "0101358072d6365880d1aeea329adf9121383851ed21a28e3b75e965d0d2cd166254174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd500100000000000000000",
"f1" to "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d77e7c60745d6ff2d29cb31ea1ee07d57dcbf42a3b1ac4341763a6f7baaa5766561e2fd98b7e4a69846548854a72be3b036eac464732b81a97afd4ef73c75f0a45703fd77c5ee8d29188ea452d9125e0bd8b798c212fccb1a65b4b617c1ebffd3c10256bc31f2949c6095f8cbc241d43b12f1591d01df46548",
"h1" to "602f30edcbbd47efaeaf6c79e30d0a418ac5dd0f18f03148dc55d6f1115ac6c1",
"replyIv" to "000102030405060708090a0b0c0d0e0f1011121314151617",
"replyRoot" to "7b747b6d95ad2bbb29d75ffd307291a1fbe9ac30e29fd8723ad859c7edd9cd50",
"serverCore" to "010179a631eede1bf9c98f12032cdeadd0e7a079398fc786b88cc846ec89af85a51a56a735db2bdb65b5240a7c7265917a909c1515ca682dcce740c9f00ac91eabcd174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd500100000000000000000",
"serverSignature" to "c9449226a851142979083bb1c633005c6323be2c868b2c188db04e63c9dd8952f1bfb549b4a43fa485cf456fe139206f92adf8a738e1ca4d95f03b0378d22906",
"f2" to "000102030405060708090a0b0c0d0e0f10111213141516179e23f0972e103c06d5b5a374b83490d3746d51aeb4efb24829822dc8a6826e417734a39ad43437e12d7e379cf633d2397a9ede00b477308a59f262f12c0804f79b7b00b86f04426c7349d53d4935601dad6eb81a95e0bbdab3d73607a6187e2bd18a83d90c93858f17dee1753cfb56395c505a004fc1cbcc2ca6c3820c81b3128c8f69138e7435dfba773b8a05cfa7a328d4bc76dd0db0aba878718c48798ee60a95cb46d24d87003a688e0eb3761871f33621b245f70d1152358cbc41e7b073de23d41579f81883725035a46248c1c17a59644e43ac9e6a",
"h2" to "ccb3ed241a73bd60911c36356a70651e8ab884ed5958514122f35d382d0d2533",
"sessionShared" to "04c304fb1ca83cee75e206344231f33797e07d9929db670994b7c6fbeb1dc255",
"s2c" to "46d6c3a94b432fab260e6252e53b488c6ce8b5dca9d17499facac2e402f1ccb1",
"c2s" to "2839576c1187d37c4e4d330ce4331c305c9ebf1ac599110fd1e9c44c1f64a5f3",
"token" to "b6e3c9512701a9edea03d60c5f50ac894bf652c776e3636f642a319357148481",
"finishedC2s" to "4e6ed04a46d1f01c64873c0050afb167b110de66ae8ab97344e79d0c527bd8ad",
"finishedS2c" to "f6de33390f63d704bf2fa82dac7aa1e21a08da48351326047d68c6dd6c9d280e",
"clientSignature" to "dc405b92ddf7499d47f3a32e837bed640d4565632e4f332ccbc7b6159a0afd1e1272bd36531a6b5d8a40c60e4a307b8a8e8aca62ba4d635ba278b19e5c448004",
"ackCore" to "03ccb3ed241a73bd60911c36356a70651e8ab884ed5958514122f35d382d0d2533cd14b37f956e953194ff7fb73b3d81dcc561d61a7538094b7c3e1a643ee5f3aadc405b92ddf7499d47f3a32e837bed640d4565632e4f332ccbc7b6159a0afd1e1272bd36531a6b5d8a40c60e4a307b8a8e8aca62ba4d635ba278b19e5c448004",
"ackIv" to "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f",
"f3" to "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f13d609bbf798a1056e5bca20eb89872e2160808864b58cf718c89dabe170b453ccfff09a287502ab1167011f79bbcdabdd673a8e26b22f4f7e7d91ec802dc1b8917820d93a24faaa86abfd0c04ec6c670a5938ac8e003aa7378e6a995b3039e7d5d1a0dd233c8589feb771ee1a6df8d8e43a6ace5ce146ca440eaaa0c3ff0cfe190be2dd60a9c129ae4cb5e3802fd306e1086a180605c59c01e1a17ffb943e517a6516f409159b1b6a9ce95079",
"h3" to "4f21ca3091f37bf39d78fea6cb5ea6dca91f0fd78442beb6874fc6a9097be5c0",
"acceptCore" to "044f21ca3091f37bf39d78fea6cb5ea6dca91f0fd78442beb6874fc6a9097be5c0",
"acceptIv" to "303132333435363738393a3b3c3d3e3f4041424344454647",
"f4" to "303132333435363738393a3b3c3d3e3f40414243444546479c1359d7f05d126e96c469773c8a586c711742c1547137a75fed8f3e29c2bb51cf8e86698471e4e4ab58e325efe7c451c6183ae167602430d9a9773baec43dccd6d4d4c067b012838fabaf7624",
"c2sStreamKey" to "3849e6ec55adeb429a2f1152fe1d6499f3c331d0c20f961ffb07d37d0a1f6ba6",
"c2sStreamSalt" to "124c709383f7fec219af21b040cafa2c28ee23b58bdbd36e",
"c2sMacKey" to "098146c2016cc5d4799e804784c424746b75b4f0dfe5ad6772147f48424cfafe",
"c2sUdpKey" to "e39c67192f6668ff5e3fe2b3155d161c30327d855d1ad7584337ee0eed3d2d85",
"s2cStreamKey" to "7d859e27ebf54f5117dc173919f4952ca4a79d449b39ca4cb66dc982ad6df43c",
"s2cStreamSalt" to "98f157f145f68670eafcc2a1e114ae269d5efe1fbc89d806",
"s2cMacKey" to "902cce94c7e7335027c3ed5ac695e6ba9ebfef8a2d8dd94bd7bf522b2f78a185",
"s2cUdpKey" to "fb470d7c6dd2980c070303660e7d11726a3b92fab057b5d6a2f63a75fc1b9453",
"object0" to "000102",
"udp0" to "000102030405060708090a0b0c0d0e0f101112131415161752519f6c14b9af07c4ff6e4506bded91385f473d499ae7a1bd77ff90835751",
"object1" to "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445",
"udp1" to "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f27e785c70cb414797285f7ece74d227f7c8773bafdc1fc59c19631825e48b1352adee3f0c8a9a61c1d49c40f6207fde77ccc279982109f22b1b33ce02f71ffe3565d6c5aa656d58caa8aded51d35c6cef5aefd5b8c08b7c892901bfbfad7c503f7ec",
"object2" to "010000",
"udp2" to "303132333435363738393a3b3c3d3e3f4041424344454647781578e7e4cdfc68bec534773c724439f04787b1f981d09987320ac7637f21",
"tcp" to "7076dc0a75456996a85759de0c9e1ddc7f19e4f42f8d21b9c67909f4e106a0388494a18df501d78b8e51bf3affa6cd10bff8307afc5b4c45494a0e89d2d11a823a7257f7d52c0379d3333e33753933630e270d5fa61f11106d80095feff564231280114d47d12d9af30825d86bb1bbf6339c0047f65976ec0c370230fab18a2f7d79ea93d848b069f0a99b42dc6bf2b6ef96f498",
)
}
@@ -0,0 +1,111 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.keyexchange.KeyExchange
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.VerifyingPublicKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import net.sergeych.kiloparsec.concealed.elligator.ExperimentalElligator25519
import net.sergeych.kiloparsec.concealed.record.*
import kotlin.test.*
/** All expected bytes originate in the standalone Python composition reference. */
class IndependentWireVectorsTest {
private fun f(name: String) = IndependentWireFixtures.bytes(name)
private fun eq(name: String, actual: UByteArray) = assertContentEquals(f(name), actual, name)
private fun envelope(root: UByteArray, context: UByteArray, core: UByteArray, tail: UByteArray = ubyteArrayOf()) =
FlightEnvelope(root, context, 16384).seal(vector(core) + tail)
@Test fun independentFourFlightEncodingSignaturesAndSessionMaterial() = runTest {
initCrypto()
val vs = SigningSecretKey.fromSeed(f("serverSigningSeed"))
val vc = SigningSecretKey.fromSeed(f("clientSigningSeed"))
eq("serverIdentity", vs.verifyingKey.keyBytes)
eq("clientIdentity", vc.verifyingKey.keyBytes)
val bootstrap = ScalarMultiplication.scalarMultiplication(f("serverSecret"), ExperimentalElligator25519.decode(f("representative")))
eq("bootstrapShared", bootstrap)
val context = f("representative") + f("heloIv") + f("serverPublic")
val root = derive(32, bootstrap, "bootstrap", context)
eq("bootstrapRoot", root)
val f1 = f("representative") + f("heloIv") + envelope(root, hash(context), f("heloCore"))
eq("f1", f1)
eq("h1", hash(label("helo") + vector(f1)))
val replyRoot = derive(32, f("clientDhPublic"), "server-reply", f("h1") + f("serverPublic") + vs.verifyingKey.keyBytes)
eq("replyRoot", replyRoot)
val signedServer = label("kiloparsec/concealed/1/server-auth") + f("h1") + vector(f("serverCore"))
eq("serverSignature", vs.sign(signedServer))
assertTrue(vs.verifyingKey.verify(f("serverSignature"), signedServer))
val f2 = f("replyIv") + envelope(replyRoot, hash(label("server-flight") + f("h1") + f("replyIv")), f("serverCore"), f("serverSignature"))
eq("f2", f2)
eq("h2", hash(label("handshake") + vector(f1) + vector(f2)))
val keys = KeyExchange.clientSessionKeys(f("clientDhPublic"), f("clientDhSecret"), f("serverDhPublic"))
val serverKeys = KeyExchange.serverSessionKeys(f("serverDhPublic"), f("serverDhSecret"), f("clientDhPublic"))
eq("c2s", keys.sendKey); eq("s2c", keys.receiveKey)
eq("c2s", serverKeys.receiveKey); eq("s2c", serverKeys.sendKey)
val token = derive(32, keys.sendKey + keys.receiveKey, "connection-token", f("h2"))
eq("token", token)
val finishC = derive(32, keys.sendKey, "finished/c2s", f("h2"))
val finishS = derive(32, keys.receiveKey, "finished/s2c", f("h2"))
eq("finishedC2s", finishC); eq("finishedS2c", finishS)
val signedClient = label("kiloparsec/concealed/1/client-auth") + token + f("h2") + vc.verifyingKey.keyBytes
eq("clientSignature", vc.sign(signedClient))
assertTrue(vc.verifyingKey.verify(f("clientSignature"), signedClient))
val f3 = f("ackIv") + envelope(finishC, hash(label("ack") + f("h2") + f("ackIv")), f("ackCore"))
eq("f3", f3)
eq("h3", hash(label("authenticated-handshake") + f("h2") + vector(f3)))
eq("f4", f("acceptIv") + envelope(finishS, hash(label("accept") + f("h2") + f("acceptIv")), f("acceptCore")))
val bounds = PeerBounds(1048576, 0, 0)
val session = makeSession(keys, true, f("h2"), f("h3"), vs.verifyingKey, bounds, bounds)
eq("token", session.token)
for ((prefix, direction) in listOf("c2s" to session.sending, "s2c" to session.receiving)) {
eq(prefix + "StreamKey", direction.streamKey); eq(prefix + "StreamSalt", direction.streamSalt)
eq(prefix + "MacKey", direction.macKey); eq(prefix + "UdpKey", direction.udpKey)
}
}
@Test fun realServerAcceptsIndependentHelloAndBindsRepresentativeAlias() = runTest {
initCrypto()
val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
val identity = SigningSecretKey.fromSeed(f("serverSigningSeed"))
val frame = f("f1")
assertEquals(frame.size, ServerHandshake.helloSize(frame.copyOfRange(0, 76), f("serverSecret")))
val server = ServerHandshake(f("serverSecret"), identity, ConcealedCarrier.Kind.STREAM, options, frame)
eq("clientDhPublic", server.clientDhPublicKey)
assertEquals(240, server.response.size)
server.destroy()
for (bit in listOf(64, 128)) {
val alias = frame.copyOf().also { it[31] = (it[31].toInt() xor bit).toUByte() }
assertFailsWith<UnauthenticatedFlight> { ServerHandshake(f("serverSecret"), identity, ConcealedCarrier.Kind.STREAM, options, alias) }
}
}
@Test fun independentContinuousStreamAndReorderedDatagrams() = runTest {
initCrypto()
val limits = RecordLimits(maxPaddingBytes = 0)
val encoder = TcpRecordEncoder(f("c2sStreamKey"), f("c2sMacKey"), f("c2sStreamSalt"), limits)
val wire = (0..2).map { encoder.encode(f("object$it")) }.fold(ubyteArrayOf()) { a, b -> a + b }
// Object record boundaries are 27, 121, 148: all deliberately nonaligned to 64 bytes.
eq("tcp", wire)
for (step in listOf(1, 3, 63, 64, 65, wire.size)) {
val decoder = TcpRecordDecoder(f("c2sStreamKey"), f("c2sMacKey"), f("c2sStreamSalt"), limits)
val actual = mutableListOf<UByteArray>()
var cursor = 0
while (cursor < wire.size) {
val end = minOf(cursor + step, wire.size)
actual += decoder.feed(wire.copyOfRange(cursor, end)); cursor = end
}
decoder.finish()
assertEquals(3, actual.size)
actual.forEachIndexed { i, value -> eq("object$i", value) }
}
val udp = UdpRecordDecoder(f("c2sUdpKey"), limits)
for (q in listOf(2, 0, 1)) eq("object$q", assertNotNull(udp.decode(f("udp$q"))))
for (q in 0..2) assertNull(udp.decode(f("udp$q")))
val damaged = f("udp0").also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() }
assertFailsWith<UnauthenticatedDatagram> { udp.decode(damaged) }
}
}
@@ -0,0 +1,191 @@
@file:OptIn(ExperimentalUnsignedTypes::class)
package net.sergeych.kiloparsec.concealed.record
import com.ionspin.kotlin.crypto.generichash.GenericHash
import com.ionspin.kotlin.crypto.secretbox.SecretBox
import com.ionspin.kotlin.crypto.stream.Stream
import kotlinx.coroutines.test.runTest
import net.sergeych.crypto2.initCrypto
import kotlin.random.Random
import kotlin.test.*
class ProtectedRecordsTest {
private val key = UByteArray(32) { it.toUByte() }
private val mac = UByteArray(32) { (it + 32).toUByte() }
private val nonce = UByteArray(24) { (it + 64).toUByte() }
private val unpadded = RecordLimits(maxPaddingBytes = 0)
@Test fun destructionPermanentlyRejectsEncoderDecoderAndCipherReuse() = runTest {
initCrypto()
val streamEncoder = TcpRecordEncoder(key, mac, nonce, unpadded)
val stream = streamEncoder.encode(ubyteArrayOf(1u))
val streamDecoder = TcpRecordDecoder(key, mac, nonce, unpadded)
assertTrue(streamDecoder.feed(stream.copyOfRange(0, 7)).isEmpty())
assertTrue(streamDecoder.hasPartialRecord)
streamEncoder.destroy(); streamDecoder.destroy()
streamEncoder.destroy(); streamDecoder.destroy() // Idempotent cleanup.
assertFalse(streamDecoder.hasPartialRecord)
assertFailsWith<IllegalStateException> { streamEncoder.encode(ubyteArrayOf()) }
assertFailsWith<IllegalStateException> { streamDecoder.feed(ubyteArrayOf()) }
assertFailsWith<IllegalStateException> { streamDecoder.finish() }
val datagramEncoder = UdpRecordEncoder(key, unpadded)
val datagram = datagramEncoder.encode(ubyteArrayOf(2u)).bytes
val datagramDecoder = UdpRecordDecoder(key, unpadded)
datagramEncoder.destroy(); datagramDecoder.destroy()
datagramEncoder.destroy(); datagramDecoder.destroy()
assertFailsWith<IllegalStateException> { datagramEncoder.encode(ubyteArrayOf()) }
assertFailsWith<IllegalStateException> { datagramDecoder.decode(datagram) }
val cipher = ContinuousCipher(key, nonce)
cipher.apply(ubyteArrayOf(1u)) // Leaves a partially used secret mask.
cipher.destroy(); cipher.destroy()
assertFailsWith<IllegalStateException> { cipher.apply(ubyteArrayOf()) }
}
@Test fun macMatchesIndependentBlake2bVector() = runTest {
initCrypto()
val body = ubyteArrayOf(0u, 0u, 0u, 3u, 1u, 0x61u, 0x62u, 0u, 0u)
val expected = "d0809fec300f48fb92d8eacd48284a8f".chunked(2).map { it.toUByte(16) }.toUByteArray()
assertContentEquals(expected, recordMac(key, 7uL, u32(body.size), body))
}
@Test fun usageCeilingsRejectBeforeRecordOrReplayMutation() = runTest {
initCrypto()
val one = RecordLimits(maxPaddingBytes = 0, maxRecords = 1uL)
val encoder = TcpRecordEncoder(key, mac, nonce, one)
val first = encoder.encode(ubyteArrayOf(1u))
assertFails { encoder.encode(ubyteArrayOf(2u)) }
val decoder = TcpRecordDecoder(key, mac, nonce, one)
assertContentEquals(ubyteArrayOf(1u), decoder.feed(first).single())
assertFailsWith<InvalidRecord> { decoder.feed(ubyteArrayOf(0u)) }
val tiny = RecordLimits(maxPaddingBytes = 0, maxStreamBytes = 24uL)
val smallEncoder = TcpRecordEncoder(key, mac, nonce, tiny)
assertFails { smallEncoder.encode(ubyteArrayOf(1u)) }
val exact = smallEncoder.encode(ubyteArrayOf())
assertContentEquals(ubyteArrayOf(), TcpRecordDecoder(key, mac, nonce, tiny).feed(exact).single())
assertFailsWith<InvalidRecord> { TcpRecordDecoder(key, mac, nonce, tiny).feed(first) }
val udp = UdpRecordEncoder(key, one)
val datagram = udp.encode(ubyteArrayOf(1u))
assertFails { udp.encode(ubyteArrayOf(2u)) }
val receiver = UdpRecordDecoder(key, one)
assertContentEquals(ubyteArrayOf(1u), receiver.decode(datagram.bytes))
assertNull(receiver.decode(datagram.bytes))
}
@Test fun streamMatchesSingleSodiumCallAcrossEveryMaskBoundary() = runTest {
initCrypto()
val input = UByteArray(4099) { (it * 7).toUByte() }
val derivedKey = GenericHash.genericHash(
"kiloparsec/concealed/stream-key/v1".encodeToByteArray().asUByteArray() + nonce, 32, key)
val derivedNonce = GenericHash.genericHash(
"kiloparsec/concealed/stream-nonce/v1".encodeToByteArray().asUByteArray() + nonce, 16, key).copyOf(8)
val expected = Stream.chacha20XorIc(input, derivedNonce, 0uL, derivedKey)
for (step in listOf(1, 3, 63, 64, 65, 127, 256)) {
val cipher = ContinuousCipher(key, nonce)
val actual = UByteArray(input.size)
var offset = 0
while (offset < input.size) {
val end = minOf(input.size, offset + step)
cipher.apply(input.copyOfRange(offset, end)).copyInto(actual, offset)
offset = end
}
assertContentEquals(expected, actual, "segmentation size $step")
}
}
@Test fun tcpSegmentationCoalescingAndVariablePadding() = runTest {
initCrypto()
val encoder = TcpRecordEncoder(key, mac, nonce)
val objects = listOf(ubyteArrayOf(), ubyteArrayOf(1u), UByteArray(511) { it.toUByte() }, UByteArray(10000))
val encoded = objects.map(encoder::encode).fold(ubyteArrayOf()) { a, b -> a + b }
for (step in listOf(1, 17, 64, 8192, encoded.size)) {
val decoder = TcpRecordDecoder(key, mac, nonce)
val actual = mutableListOf<UByteArray>()
var offset = 0
while (offset < encoded.size) {
val end = minOf(offset + step, encoded.size)
actual += decoder.feed(encoded.copyOfRange(offset, end))
offset = end
}
decoder.finish()
assertEquals(objects.size, actual.size)
objects.zip(actual).forEach { (a, b) -> assertContentEquals(a, b) }
}
}
@Test fun tcpRejectsEveryModifiedByteAndNeverPublishesUnauthenticatedBody() = runTest {
initCrypto()
val bytes = TcpRecordEncoder(key, mac, nonce, unpadded).encode(UByteArray(70) { it.toUByte() })
for (position in bytes.indices) {
val tampered = bytes.copyOf()
tampered[position] = (tampered[position].toInt() xor 1).toUByte()
val decoder = TcpRecordDecoder(key, mac, nonce, unpadded)
val result = runCatching { decoder.feed(tampered) }
// A manipulated length may leave a bounded partial read, but never yield an object.
if (result.isSuccess) {
assertTrue(result.getOrThrow().isEmpty())
assertFailsWith<InvalidRecord> { decoder.finish() }
}
assertFails { decoder.feed(ubyteArrayOf(0u)) }
}
}
@Test fun tcpRejectsReorderedRecordsAndTruncation() = runTest {
initCrypto()
val encoder = TcpRecordEncoder(key, mac, nonce, unpadded)
val first = encoder.encode(ubyteArrayOf(1u))
val second = encoder.encode(ubyteArrayOf(2u))
assertFails { TcpRecordDecoder(key, mac, nonce, unpadded).feed(second + first) }
for (n in 1 until first.size) {
val decoder = TcpRecordDecoder(key, mac, nonce, unpadded)
assertTrue(decoder.feed(first.copyOf(n)).isEmpty())
assertFailsWith<InvalidRecord> { decoder.finish() }
}
}
@Test fun authenticatedMalformedInnerLengthFailsBeforeObjectDelivery() = runTest {
initCrypto()
val body = u32(100) + ubyteArrayOf(1u)
val length = u32(body.size)
val cipher = ContinuousCipher(key, nonce)
val malformed = cipher.apply(length + body + recordMac(mac, 0uL, length, body))
assertFailsWith<InvalidRecord> { TcpRecordDecoder(key, mac, nonce).feed(malformed) }
}
@Test fun udpReordersDropsDuplicatesAndPreservesWindowOnForgery() = runTest {
initCrypto()
val encoder = UdpRecordEncoder(key, unpadded)
val packets = (0..4100).map { encoder.encode(u32(it)) }
val decoder = UdpRecordDecoder(key, unpadded)
assertContentEquals(u32(4100), decoder.decode(packets[4100].bytes))
assertNull(decoder.decode(packets[0].bytes))
assertContentEquals(u32(4099), decoder.decode(packets[4099].bytes))
assertNull(decoder.decode(packets[4099].bytes))
val forged = packets[4098].bytes.copyOf()
forged[forged.lastIndex] = (forged.last().toInt() xor 1).toUByte()
assertFailsWith<InvalidRecord> { decoder.decode(forged) }
assertContentEquals(u32(4098), decoder.decode(packets[4098].bytes))
}
@Test fun udpRandomReorderingRoundTripAndBudget() = runTest {
initCrypto()
val limits = RecordLimits(maxDatagramBytes = 200)
val encoder = UdpRecordEncoder(key, limits)
val decoder = UdpRecordDecoder(key, limits)
val packets = (0..100).map { encoder.encode(u32(it)) }
assertTrue(packets.all { it.bytes.size <= 200 })
packets.shuffled(Random(73)).forEach { assertContentEquals(u32(it.sequence.toInt()), decoder.decode(it.bytes)) }
assertEquals(200, encoder.encode(UByteArray(148)).bytes.size)
assertFailsWith<IllegalArgumentException> { encoder.encode(UByteArray(149)) }
}
@Test fun udpTamperedSequenceDoesNotPoisonReplayState() = runTest {
initCrypto()
val nonce = UByteArray(24)
val malformed = nonce + SecretBox.easy(u64(123uL) + u32(999), nonce, key)
val decoder = UdpRecordDecoder(key)
assertFailsWith<InvalidRecord> { decoder.decode(malformed) }
val good = nonce + SecretBox.easy(u64(123uL) + u32(1) + ubyteArrayOf(7u), nonce, key)
assertContentEquals(ubyteArrayOf(7u), decoder.decode(good))
}
}
@@ -14,6 +14,10 @@ import assertThrows
import io.ktor.server.engine.* import io.ktor.server.engine.*
import io.ktor.server.netty.* import io.ktor.server.netty.*
import kotlinx.coroutines.delay import kotlinx.coroutines.delay
import kotlinx.coroutines.Dispatchers
import kotlinx.coroutines.withContext
import kotlinx.coroutines.withTimeout
import kotlinx.coroutines.flow.first
import kotlinx.coroutines.launch import kotlinx.coroutines.launch
import kotlinx.coroutines.runBlocking import kotlinx.coroutines.runBlocking
import kotlinx.coroutines.test.runTest import kotlinx.coroutines.test.runTest
@@ -93,10 +97,10 @@ class ClientTest {
client.call(cmdClose) client.call(cmdClose)
} }
// connection should now be closed // Socket teardown uses real IO; a virtual test delay cannot wait for it.
// the problem is: it needs some unspecified time to close withContext(Dispatchers.Default) {
// as it is async process. withTimeout(5_000) { client.connectedStateFlow.first { !it } }
delay(100) }
assertFalse { client.connectedStateFlow.value } assertFalse { client.connectedStateFlow.value }
// this should be run on automatically reopen connection // this should be run on automatically reopen connection
@@ -0,0 +1,89 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.flow.collect
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.*
import net.sergeych.kiloparsec.adapter.NetworkAddress
import kotlin.test.*
class ConcealedLoopbackTest {
@Test fun actualTcpConcurrentDuplexAndPush() = runBlocking<Unit> {
val listener = ConcealedTcpListener(localInterface = "127.0.0.1")
try { checkDuplex(listener.carriers, { connectConcealedTcpCarrier(NetworkAddress("127.0.0.1", listener.boundPort())) }) }
finally { listener.close() }
}
@Test fun actualUdpConcurrentDuplexAndPush() = runBlocking<Unit> {
val listener = ConcealedUdpListener(localInterface = "127.0.0.1")
try { checkDuplex(listener.carriers, { connectConcealedUdpCarrier(NetworkAddress("127.0.0.1", listener.boundPort())) }) }
finally { listener.close() }
}
private suspend fun checkDuplex(carriers: Flow<ConcealedCarrier>, connect: suspend () -> ConcealedCarrier) = coroutineScope {
withTimeout(30_000) {
initCrypto()
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val identity = ConcealedServerIdentity(encryption, signing)
val echo by command<Int, Int>()
val notice by command<Int, Unit>()
val notices = Channel<Int>(32)
val serverLocal = KiloInterface<Unit>().apply {
on(echo) { it * 2 }
on(notice) { notices.send(it) }
onConnected { assertEquals(clientSigning.verifyingKey, remoteIdentity) }
}
val clientLocal = KiloInterface<Unit>().apply {
on(echo) { it + 100 }
on(notice) { notices.send(it) }
onConnected { assertEquals(signing.verifyingKey, remoteIdentity) }
}
val serverReady = CompletableDeferred<ConcealedConnection<Unit>>()
var serverRun: Deferred<Result<Unit>>? = null
val listenerJob = launch {
carriers.collect { carrier ->
val server = ConcealedConnection.server(serverLocal, carrier, Unit, identity)
serverReady.complete(server)
serverRun = async { runCatching { server.run() } }
}
}
val client = ConcealedConnection.client(clientLocal, connect(), Unit, encryption.publicKey,
signing.verifyingKey, clientSigning)
val clientRun = async { runCatching { client.run() } }
val early = async { client.call(echo, 7) }
var server: ConcealedConnection<Unit>? = null
try {
server = serverReady.await()
client.awaitReady(); server.awaitReady()
assertEquals(14, early.await())
assertContentEquals(client.token(), server.token())
val activeServer = server
coroutineScope {
(0 until 8).flatMap { i -> listOf(
async(Dispatchers.Default) { assertEquals(i * 2, client.call(echo, i)) },
async(Dispatchers.Default) { assertEquals(i + 100, activeServer.call(echo, i)) },
) }.awaitAll()
}
client.push(notice, 19); server.push(notice, 23)
assertEquals(setOf(19, 23), setOf(notices.receive(), notices.receive()))
assertTrue(clientRun.isActive)
assertTrue(serverRun!!.isActive)
assertNull(client.terminalFailure)
assertNull(server.terminalFailure)
client.close()
withTimeout(3000) { clientRun.await(); serverRun!!.await() }
} finally {
client.close(); server?.close()
clientRun.cancelAndJoin(); serverRun?.cancelAndJoin(); listenerJob.cancelAndJoin()
}
}
}
}
@@ -0,0 +1,183 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import java.io.File
import java.net.DatagramPacket
import java.net.DatagramSocket
import java.net.InetAddress
import java.net.InetSocketAddress
import java.net.ServerSocket
import java.net.Socket
import java.util.Collections
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.Flow
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.KiloInterface
import net.sergeych.kiloparsec.adapter.NetworkAddress
import net.sergeych.kiloparsec.command
import org.junit.Assume.assumeTrue
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFalse
import kotlin.test.assertNull
import kotlin.test.assertTrue
/** Real loopback socket payload capture through forwarding proxies; requires no packet privileges. */
class ConcealedTrafficCaptureTest {
private data class Capture(val session: Int, val direction: String, val time: Long, val bytes: ByteArray)
private val captures = Collections.synchronizedList(mutableListOf<Capture>())
private val loopback = InetAddress.getByName("127.0.0.1")
@Test fun actualSocketPayloadVisibility() = runBlocking {
assumeTrue(System.getenv("KILOPARSEC_EXTENDED_TESTS") == "1")
withTimeout(60_000) {
initCrypto()
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val identity = ConcealedServerIdentity(encryption, signing)
val echo by command<String, String>()
val marker = "SYNTHETIC-CONCEALED-RPC-MARKER-237d404-".repeat(3)
val local = KiloInterface<Unit>().apply { on(echo) { it } }
val tcp = ConcealedTcpListener(localInterface = "127.0.0.1")
val udp = ConcealedUdpListener(localInterface = "127.0.0.1")
val ready = Channel<Pair<ConcealedConnection<Unit>, Deferred<Result<Unit>>>>(1)
suspend fun collect(carriers: Flow<ConcealedCarrier>) = coroutineScope {
carriers.collect { carrier ->
val server = ConcealedConnection.server(local, carrier, Unit, identity)
val job = async { runCatching { server.run() } }
ready.send(server to job)
}
}
val tcpAccept = launch { collect(tcp.carriers) }
val udpAccept = launch { collect(udp.carriers) }
val proxies = mutableListOf<Proxy>()
try {
for (kind in ConcealedCarrier.Kind.entries) repeat(8) { index ->
val number = index + if (kind == ConcealedCarrier.Kind.STREAM) 0 else 8
val target = if (kind == ConcealedCarrier.Kind.STREAM) tcp.boundPort() else udp.boundPort()
val proxy = if (kind == ConcealedCarrier.Kind.STREAM) TcpProxy(number, target, this)
else UdpProxy(number, target, this)
proxies += proxy
val carrier = if (kind == ConcealedCarrier.Kind.STREAM)
connectConcealedTcpCarrier(NetworkAddress("127.0.0.1", proxy.port))
else connectConcealedUdpCarrier(NetworkAddress("127.0.0.1", proxy.port))
val client = ConcealedConnection.client(local, carrier, Unit, encryption.publicKey,
signing.verifyingKey, clientSigning)
val run = async { runCatching { client.run() } }
var server: ConcealedConnection<Unit>? = null
var serverRun: Deferred<Result<Unit>>? = null
try {
val accepted = ready.receive()
server = accepted.first; serverRun = accepted.second
client.awaitReady(); server.awaitReady()
repeat(4) { assertEquals(marker, client.call(echo, marker)) }
assertEquals(marker, server.call(echo, marker))
assertNull(client.terminalFailure)
assertNull(server.terminalFailure)
client.close()
withTimeout(5_000) { run.await().getOrThrow(); serverRun.await().getOrThrow() }
} finally {
client.close(); server?.close(); proxy.close()
run.cancelAndJoin(); serverRun?.cancelAndJoin()
}
}
} finally {
proxies.forEach { it.close() }
tcp.close(); udp.close()
tcpAccept.cancelAndJoin(); udpAccept.cancelAndJoin(); ready.cancel()
}
val snapshot = synchronized(captures) { captures.toList() }
assertTrue(snapshot.isNotEmpty())
val streams = snapshot.groupBy { it.session to it.direction }.mapValues { (_, parts) ->
parts.fold(ByteArray(0)) { a, p -> a + p.bytes }
}
val secrets = listOf(marker.encodeToByteArray(), signing.verifyingKey.keyBytes.asByteArray(),
clientSigning.verifyingKey.keyBytes.asByteArray(), encryption.publicKey.keyBytes.asByteArray())
streams.values.forEach { bytes -> secrets.forEach { assertFalse(contains(bytes, it)) } }
val firstPrefixes = (0 until 16).map { id -> streams.getValue(id to "c2s").take(32).toByteArray() }
assertEquals(16, firstPrefixes.map { it.joinToString() }.toSet().size)
val directory = File("build/concealed-qualification/traffic").apply { mkdirs() }
snapshot.forEachIndexed { i, c -> File(directory, "${c.session}-${c.direction}-$i.bin").writeBytes(c.bytes) }
File(directory, "capture.csv").printWriter().use { out ->
out.println("session,carrier,direction,monotonic_ns,payload_bytes,file")
snapshot.forEachIndexed { i, c -> out.println("${c.session},${if (c.session < 8) "tcp" else "udp"},${c.direction},${c.time},${c.bytes.size},${c.session}-${c.direction}-$i.bin") }
}
val report = """
Real loopback socket payload capture: 8 TCP + 8 UDP sessions, 5 duplex echo calls/session.
${snapshot.size} forwarded socket chunks/datagrams, ${snapshot.sumOf { it.bytes.size }} bytes.
All 16 first-flight representative prefixes differ. No synthetic RPC marker or static encryption/server-identity/client-identity public-key bytes appeared in either direction's captured payload stream.
TCP chunks are proxy read boundaries, not protocol-record or IP-packet boundaries. UDP captures preserve datagram boundaries. Random padding and transport segmentation produce observable sizes in capture.csv.
Payload capture only: no privileged IP packet capture, WebSocket upgrade capture, Internet path, active-probe classification or cryptographic indistinguishability proof. Endpoints, traffic volume and timing remain observable. First-flight lengths and phase structure may still support statistical classification.
""".trimIndent()
File(directory, "summary.txt").writeText(report)
println(report)
}
}
private fun contains(haystack: ByteArray, needle: ByteArray): Boolean =
(0..haystack.size - needle.size).any { start -> needle.indices.all { haystack[start + it] == needle[it] } }
private fun record(session: Int, direction: String, bytes: ByteArray) {
captures += Capture(session, direction, System.nanoTime(), bytes.copyOf())
}
private interface Proxy { val port: Int; suspend fun close() }
private inner class TcpProxy(id: Int, target: Int, scope: CoroutineScope) : Proxy {
private val listener = ServerSocket(0, 1, loopback)
override val port = listener.localPort
private val sockets = Collections.synchronizedList(mutableListOf<Socket>())
private val job = scope.launch(Dispatchers.IO) {
try {
val client = listener.accept().also { sockets += it }
val server = Socket(loopback, target).also { sockets += it }
suspend fun pipe(from: Socket, to: Socket, direction: String) {
try {
val bytes = ByteArray(8192)
while (true) {
val size = from.getInputStream().read(bytes)
if (size < 0) break
val chunk = bytes.copyOf(size)
record(id, direction, chunk)
to.getOutputStream().write(chunk); to.getOutputStream().flush()
}
runCatching { to.shutdownOutput() }
} catch (_: java.io.IOException) { /* Proxy shutdown closes blocking reads. */ }
}
joinAll(launch { pipe(client, server, "c2s") }, launch { pipe(server, client, "s2c") })
} catch (_: java.io.IOException) { /* Listener shutdown. */ }
}
override suspend fun close() {
listener.close()
synchronized(sockets) { sockets.forEach { it.close() } }
job.cancelAndJoin()
}
}
private inner class UdpProxy(id: Int, target: Int, scope: CoroutineScope) : Proxy {
private val socket = DatagramSocket(InetSocketAddress(loopback, 0))
override val port = socket.localPort
private val job = scope.launch(Dispatchers.IO) {
var client: java.net.SocketAddress? = null
try {
while (isActive) {
val packet = DatagramPacket(ByteArray(65507), 65507)
socket.receive(packet)
val fromServer = packet.address == loopback && packet.port == target
val destination = if (fromServer) client ?: continue else {
client = packet.socketAddress; InetSocketAddress(loopback, target)
}
val bytes = packet.data.copyOfRange(packet.offset, packet.offset + packet.length)
record(id, if (fromServer) "s2c" else "c2s", bytes)
socket.send(DatagramPacket(bytes, bytes.size, destination))
}
} catch (_: java.io.IOException) { /* Proxy shutdown. */ }
}
override suspend fun close() { socket.close(); job.cancelAndJoin() }
}
}
@@ -0,0 +1,111 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import io.ktor.client.HttpClient
import io.ktor.client.engine.cio.CIO
import io.ktor.client.plugins.websocket.WebSockets as ClientWebSockets
import io.ktor.client.request.get
import io.ktor.client.statement.bodyAsText
import io.ktor.server.application.install
import io.ktor.server.engine.embeddedServer
import io.ktor.server.netty.Netty
import io.ktor.server.response.respondText
import io.ktor.server.routing.get
import io.ktor.server.routing.routing
import io.ktor.server.websocket.WebSockets as ServerWebSockets
import io.ktor.server.websocket.webSocket
import io.ktor.websocket.Frame
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.*
import kotlin.test.*
import org.junit.Test as JUnitTest
class ConcealedWebSocketTest {
@JUnitTest(timeout = 45_000) fun realWebSocketConcealedHandshakeDuplexPushAndCallerClientOwnership() = runBlocking<Unit> {
withTimeout(30_000) {
initCrypto()
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val identity = ConcealedServerIdentity(encryption, signing)
val echo by command<Int, Int>()
val notice by command<Int, Unit>()
val notices = Channel<Int>(8)
val servers = Channel<ConcealedConnection<Unit>>(2)
val serverLocal = KiloInterface<Unit>().apply {
on(echo) { it * 2 }
on(notice) { notices.send(it) }
onConnected { assertEquals(clientSigning.verifyingKey, remoteIdentity) }
}
val serverScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val app = serverScope.embeddedServer(Netty, host = "127.0.0.1", port = 0) {
install(ServerWebSockets) { maxFrameSize = 257 }
routing {
get("/health") { call.respondText("alive") }
webSocket("/concealed") {
val server = ConcealedConnection.server(serverLocal, asConcealedCarrier(257), Unit, identity)
servers.send(server)
runCatching { server.run() }
}
}
}.start(wait = false)
val port = app.engine.resolvedConnectors().single().port
val external = HttpClient(CIO) { install(ClientWebSockets) { maxFrameSize = 257 } }
try {
for (callerOwned in listOf(false, true)) {
val carrier = connectConcealedWebSocketCarrier("ws://127.0.0.1:$port/concealed",
if (callerOwned) external else null, maxChunkBytes = 257)
val clientLocal = KiloInterface<Unit>().apply {
on(echo) { it + 100 }
on(notice) { notices.send(it) }
}
val client = ConcealedConnection.client(clientLocal, carrier, Unit, encryption.publicKey,
signing.verifyingKey, clientSigning)
val running = async { runCatching { client.run() } }
val server = servers.receive()
try {
client.awaitReady(); server.awaitReady()
assertContentEquals(client.token(), server.token())
coroutineScope {
(0 until 4).flatMap { i -> listOf(
async { assertEquals(i * 2, client.call(echo, i)) },
async { assertEquals(i + 100, server.call(echo, i)) },
) }.awaitAll()
}
client.push(notice, 7); server.push(notice, 11)
assertEquals(setOf(7, 11), setOf(notices.receive(), notices.receive()))
assertNull(client.terminalFailure)
} finally { client.close(); server.close(); running.cancelAndJoin() }
}
assertEquals("alive", external.get("http://127.0.0.1:$port/health").bodyAsText())
} finally { external.close(); app.stop(0, 1000); serverScope.cancel() }
}
}
@JUnitTest(timeout = 15_000) fun websocketTextIsRejectedInsteadOfInterpretedAsBase64() = runBlocking<Unit> {
withTimeout(5_000) {
val serverScope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
val app = serverScope.embeddedServer(Netty, host = "127.0.0.1", port = 0) {
install(ServerWebSockets) { maxFrameSize = 1024 }
routing {
webSocket("/concealed") {
send(Frame.Text("AQID"))
flush()
// Observe peer closure so the route exits; an unconditional suspension
// here survives the test body's cancellation and leaks a server child job.
for (ignored in incoming) { /* no application payload expected */ }
}
}
}.start(wait = false)
val port = app.engine.resolvedConnectors().single().port
val carrier = connectConcealedWebSocketCarrier("ws://127.0.0.1:$port/concealed", maxChunkBytes = 1024)
try { assertFailsWith<IllegalStateException> { carrier.receive() } }
finally { carrier.close(); app.stop(0, 1000); serverScope.cancel() }
}
}
}
@@ -0,0 +1,252 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import com.ionspin.kotlin.crypto.secretbox.SecretBox
import java.util.concurrent.atomic.AtomicInteger
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.*
import net.sergeych.kiloparsec.adapter.PriorityBlockWriter
import net.sergeych.kiloparsec.concealed.record.*
import org.junit.Assume.assumeTrue
import kotlin.random.Random
import kotlin.test.*
/** Opt-in real-time qualification, deliberately excluded from the normal fast feedback loop. */
class ExtendedConcealedQualificationTest {
private val key = UByteArray(32) { it.toUByte() }
private val mac = UByteArray(32) { (it + 32).toUByte() }
private val nonce = UByteArray(24) { (it + 64).toUByte() }
private val limits = RecordLimits(maxBodyBytes = 512, maxPaddingBytes = 0, maxDatagramBytes = 512)
private fun enabled() = assumeTrue("Set KILOPARSEC_EXTENDED_TESTS=1", System.getenv("KILOPARSEC_EXTENDED_TESTS") == "1")
@Test fun tenThousandDeterministicMalformedRecordsNeverReleaseData() = runBlocking {
enabled(); initCrypto()
withTimeout(60_000) {
val random = Random(0x237d404)
val valid = TcpRecordEncoder(key, mac, nonce, limits).encode(UByteArray(128) { it.toUByte() })
val udp = UdpRecordDecoder(key, limits)
val udpEncoder = UdpRecordEncoder(key, limits)
val classes = IntArray(4)
repeat(10_000) { case ->
ensureActive()
val kind = case % 4
classes[kind]++
when (kind) {
0 -> {
// An arbitrary unauthenticated datagram may never alter replay state.
val bytes = ByteArray(random.nextInt(0, 600)).also { random.nextBytes(it) }.asUByteArray()
assertFailsWith<UnauthenticatedDatagram>("random UDP case $case") { udp.decode(bytes) }
}
1 -> {
// Genuine tag with inconsistent inner size: authenticated semantic failure.
val iv = ByteArray(24).also { random.nextBytes(it) }.asUByteArray()
val forged = iv + SecretBox.easy(u64(case.toULong()) + u32(513) + ubyteArrayOf(1u), iv, key)
assertFailsWith<InvalidRecord>("authenticated UDP case $case") { udp.decode(forged) }
}
else -> {
val decoder = TcpRecordDecoder(key, mac, nonce, limits)
try {
val bytes = if (kind == 2) valid.copyOf().also {
val at = random.nextInt(it.size)
it[at] = (it[at].toInt() xor (1 shl random.nextInt(8))).toUByte()
} else valid.copyOf(random.nextInt(1, valid.size))
// Segmentation is independent of the fault position.
val result = runCatching {
var offset = 0
while (offset < bytes.size) {
val end = minOf(bytes.size, offset + random.nextInt(1, 33))
assertTrue(decoder.feed(bytes.copyOfRange(offset, end)).isEmpty(), "TCP case $case")
offset = end
}
decoder.finish()
}
assertTrue(result.isFailure, "TCP case $case must fail authentication or EOF")
assertTrue(result.exceptionOrNull() is InvalidRecord, "case $case: ${result.exceptionOrNull()}")
assertFailsWith<IllegalStateException> { decoder.feed(ubyteArrayOf(1u)) }
} finally { decoder.destroy() }
}
}
if (case % 100 == 0) {
val good = udpEncoder.encode(u32(case))
assertContentEquals(u32(case), udp.decode(good.bytes), "Forgery must not advance replay state")
assertNull(udp.decode(good.bytes))
}
}
udp.destroy(); udpEncoder.destroy()
assertContentEquals(intArrayOf(2500, 2500, 2500, 2500), classes)
println("QUALIFICATION malformed=10000 seed=0x237d404 classes=${classes.joinToString()}; no unauthenticated delivery")
}
}
@Test fun seededUdpLossReorderingDuplicatesAndForgeryRemainBounded() = runBlocking {
enabled(); initCrypto()
withTimeout(30_000) {
val random = Random(0x51a7)
val encoder = UdpRecordEncoder(key, limits)
val decoder = UdpRecordDecoder(key, limits)
var delivered = 0
var duplicateDrops = 0
var intentionallyLost = 0
var firstLost: UByteArray? = null
try {
repeat(100) { batch ->
val packets = (0 until 128).map { index -> encoder.encode(u32(batch * 128 + index)) }
val expected = packets.filter { it.sequence % 11uL != 0uL }.map { it.sequence.toInt() }.toSet()
intentionallyLost += 128 - expected.size
if (firstLost == null) firstLost = packets.first().bytes
val received = mutableSetOf<Int>()
for (packet in packets.shuffled(random)) {
if (packet.sequence % 11uL == 0uL) continue
val forged = packet.bytes.copyOf().also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() }
assertFailsWith<UnauthenticatedDatagram> { decoder.decode(forged) }
val payload = assertNotNull(decoder.decode(packet.bytes))
val id = readU32(payload)
assertTrue(received.add(id), "Duplicate execution of sequence $id")
delivered++
if (packet.sequence % 5uL == 0uL) {
assertNull(decoder.decode(packet.bytes)); duplicateDrops++
}
}
assertEquals(expected, received, "batch $batch")
}
assertNull(decoder.decode(assertNotNull(firstLost)), "Loss beyond replay window must remain stale")
assertEquals(12_800, delivered + intentionallyLost)
assertTrue(duplicateDrops > 2_000)
println("QUALIFICATION UDP seed=0x51a7 generated=12800 delivered=$delivered lost=$intentionallyLost duplicates=$duplicateDrops; bounded 128-packet shuffle")
} finally { encoder.destroy(); decoder.destroy() }
}
}
@Test fun invalidSourceFloodCannotExceedAdmissionOrSourceTableBudgets() {
enabled()
var clock = 0.0
val budgets = NewCarrierAdmission(ConcealedAdmissionLimits(
globalPerSecond = 100, globalBurst = 200, perIpPerSecond = 10, perIpBurst = 20,
maxSourceEntries = 64, sourceIdleMillis = 60_000), { clock })
var accepted = 0
repeat(10_000) { if (budgets.admit("source-${it % 64}")) accepted++ }
assertEquals(200, accepted)
clock = 1_000.0
// All 64 sources were recorded, so a rotating source cannot grow the table.
repeat(1_000) { assertFalse(budgets.admit("new-$it")) }
var refilled = 0
repeat(10_000) { if (budgets.admit("source-${it % 64}")) refilled++ }
assertEquals(100, refilled)
clock = 62_000.0
assertTrue(budgets.admit("after-idle"))
println("QUALIFICATION admission: 10000 attempts, burst=$accepted, next-second=$refilled, source-table=64, idle-expiry verified")
}
@Test fun sixtySecondsDuplexPriorityLoadAndQueueSaturationTerminateCleanly() = runBlocking {
enabled(); initCrypto()
withTimeout(100_000) {
withContext(Dispatchers.Default) {
verifyQueueBounds()
val a = Channel<UByteArray>(16)
val b = Channel<UByteArray>(16)
val clientCarrier = BoundedCarrier(a, b)
val serverCarrier = BoundedCarrier(b, a)
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val echo by command<Int, Int>()
val low by command<Int, Unit>()
val lowReceived = AtomicInteger()
val local = KiloInterface<Unit>().apply {
on(echo) { it xor 0x55aa }
on(low) { lowReceived.incrementAndGet(); Unit }
}
val options = ConcealedOptions(padding = ConcealedOptions.Padding(31),
queue = ConcealedOptions.Queue(maxItems = 16, maxBytes = 8192, maxOvertaking = 8))
val client = ConcealedConnection.client(local, clientCarrier, Unit, encryption.publicKey,
signing.verifyingKey, clientSigning, options)
val server = ConcealedConnection.server(local, serverCarrier, Unit,
ConcealedServerIdentity(encryption, signing), options)
val clientRun = async { runCatching { client.run() } }
val serverRun = async { runCatching { server.run() } }
val completions = Array(8) { AtomicInteger() }
var workers = emptyList<Job>()
try {
client.awaitReady(); server.awaitReady()
workers = completions.indices.map { index -> launch {
val remote = if (index % 2 == 0) client else server
var next = index
while (isActive) {
assertEquals(next xor 0x55aa, remote.call(echo, next, priority = if (index == 0) -100 else 100))
completions[index].incrementAndGet()
next += 8
}
} } + launch {
var next = 0
while (isActive) { client.push(low, next++, priority = -100); delay(100) }
}
var previousLow = 0
var previousLowRpc = 0
repeat(6) { interval ->
delay(10_000)
assertTrue(clientRun.isActive && serverRun.isActive)
val currentLow = lowReceived.get()
assertTrue(currentLow > previousLow, "Low-priority push stalled in interval $interval")
val currentLowRpc = completions[0].get()
assertTrue(currentLowRpc > previousLowRpc, "Low-priority RPC stalled in interval $interval")
previousLow = currentLow
previousLowRpc = currentLowRpc
}
workers.forEach { it.cancelAndJoin() }; workers = emptyList()
assertTrue(completions.all { it.get() >= 20 }, "Duplex workers: ${completions.map { it.get() }}")
assertTrue(lowReceived.get() >= 30)
assertNull(client.terminalFailure); assertNull(server.terminalFailure)
client.close()
withTimeout(5_000) { clientRun.await(); serverRun.await() }
println("QUALIFICATION load=60s workers=8 calls=${completions.map { it.get() }} lowPush=${lowReceived.get()} queue=16/8192 carrier=16; clean bounded termination")
} finally {
workers.forEach { it.cancelAndJoin() }
client.close(); server.close()
clientRun.cancelAndJoin(); serverRun.cancelAndJoin()
}
}
}
}
private suspend fun verifyQueueBounds() = coroutineScope {
val entered = CompletableDeferred<Unit>()
val release = CompletableDeferred<Unit>()
val writer = PriorityBlockWriter(this, maxQueuedItems = 4, maxQueuedBytes = 32,
maxOvertaking = 2) { entered.complete(Unit); release.await() }
try {
val first = writer.enqueue(UByteArray(8), 0uL, 0)
entered.await()
val queued = (1..4).map { writer.enqueue(UByteArray(8), it.toULong(), 100) }
assertFailsWith<RemoteInterface.Exception> { writer.enqueue(UByteArray(1), 5uL, 100) }
release.complete(Unit)
first(); queued.forEach { it() }; writer.flush()
} finally { release.complete(Unit); writer.close() }
val byteEntered = CompletableDeferred<Unit>()
val byteRelease = CompletableDeferred<Unit>()
val byteWriter = PriorityBlockWriter(this, maxQueuedItems = 8, maxQueuedBytes = 16) {
byteEntered.complete(Unit); byteRelease.await()
}
try {
val first = byteWriter.enqueue(UByteArray(1))
byteEntered.await()
val queued = byteWriter.enqueue(UByteArray(16))
assertFailsWith<RemoteInterface.Exception> { byteWriter.enqueue(UByteArray(1)) }
byteRelease.complete(Unit); first(); queued(); byteWriter.flush()
} finally { byteRelease.complete(Unit); byteWriter.close() }
}
private class BoundedCarrier(
private val incoming: Channel<UByteArray>,
private val outgoing: Channel<UByteArray>,
) : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.STREAM
override suspend fun send(bytes: UByteArray) { delay(1); outgoing.send(bytes.copyOf()) }
override suspend fun receive(): UByteArray? = incoming.receiveCatching().getOrNull()
override suspend fun close() { incoming.close(); outgoing.close() }
}
}
@@ -0,0 +1,138 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import java.net.DatagramPacket
import java.net.DatagramSocket
import java.net.InetAddress
import java.net.ServerSocket
import java.net.Socket
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.collect
import net.sergeych.kiloparsec.adapter.NetworkAddress
import kotlin.test.*
class RawConcealedCarriersTest {
private val bytes = byteArrayOf(0, 1, 127, -128, -1, 42)
@Test fun admissionBoundsGlobalPerIpAndSourceTableAndExpiresIdleEntries() {
var clock = 0.0
val budgets = NewCarrierAdmission(ConcealedAdmissionLimits(
globalPerSecond = 3, globalBurst = 3, perIpPerSecond = 2, perIpBurst = 2,
maxSourceEntries = 2, sourceIdleMillis = 2000), { clock })
assertTrue(budgets.admit("a"))
assertTrue(budgets.admit("a"))
assertFalse(budgets.admit("a")) // per-IP burst exhausted
assertTrue(budgets.admit("b"))
assertFalse(budgets.admit("b")) // global burst exhausted
clock = 1000.0
assertFalse(budgets.admit("c")) // bounded table has no new source capacity
assertTrue(budgets.admit("a")) // tokens replenished
clock = 3100.0
assertTrue(budgets.admit("c")) // inactive entries expired
}
@Test fun admissionPartialRefillDoesNotManufactureBurstTokens() {
var clock = 0.0
val budgets = NewCarrierAdmission(ConcealedAdmissionLimits(
globalPerSecond = 100, globalBurst = 100, perIpPerSecond = 10, perIpBurst = 1), { clock })
assertTrue(budgets.admit("a"))
clock = 50.0
assertFalse(budgets.admit("a"))
clock = 100.0
assertTrue(budgets.admit("a"))
assertFalse(budgets.admit("a"))
}
@Test fun tcpListenerExchangesExactlyRawBytesAndCloseUnblocksRead() = runBlocking<Unit> {
withTimeout(10_000) {
val listener = ConcealedTcpListener(localInterface = "127.0.0.1")
val accepted = CompletableDeferred<ConcealedCarrier>()
val collector = launch { listener.carriers.collect { accepted.complete(it) } }
val raw = withContext(Dispatchers.IO) { Socket("127.0.0.1", listener.boundPort()).also { it.soTimeout = 3000 } }
try {
val carrier = accepted.await()
assertEquals(ConcealedCarrier.Kind.STREAM, carrier.kind)
withContext(Dispatchers.IO) { raw.getOutputStream().write(bytes) }
assertContentEquals(bytes, readExactly(carrier, bytes.size).asByteArray())
carrier.send(bytes.asUByteArray())
val received = withContext(Dispatchers.IO) { raw.getInputStream().readNBytes(bytes.size) }
assertContentEquals(bytes, received)
val waiting = async { runCatching { carrier.receive() } }
listener.close()
waiting.await()
collector.join()
} finally { raw.close(); listener.close(); collector.cancelAndJoin() }
}
}
@Test fun tcpConnectorHasNoLegacyPrefix() = runBlocking<Unit> {
withTimeout(10_000) {
val rawServer = ServerSocket(0, 8, InetAddress.getByName("127.0.0.1"))
val accept = async(Dispatchers.IO) { rawServer.accept().also { it.soTimeout = 3000 } }
val carrier = connectConcealedTcpCarrier(NetworkAddress("127.0.0.1", rawServer.localPort))
val raw = accept.await()
try {
carrier.send(bytes.asUByteArray())
assertContentEquals(bytes, withContext(Dispatchers.IO) { raw.getInputStream().readNBytes(bytes.size) })
withContext(Dispatchers.IO) { raw.getOutputStream().write(bytes) }
assertContentEquals(bytes, readExactly(carrier, bytes.size).asByteArray())
} finally { carrier.close(); raw.close(); rawServer.close() }
}
}
@Test fun udpListenerPreservesRawDatagramsAndBoundsSourceCarriers() = runBlocking<Unit> {
withTimeout(10_000) {
val listener = ConcealedUdpListener(localInterface = "127.0.0.1",
limits = ConcealedUdpListenerLimits(maxCarriers = 1, maxCarriersPerIp = 1))
val receivedCarriers = kotlinx.coroutines.channels.Channel<ConcealedCarrier>(2)
val collector = launch { listener.carriers.collect { receivedCarriers.send(it) } }
val raw = DatagramSocket().also { it.soTimeout = 3000 }
val second = DatagramSocket()
val address = InetAddress.getByName("127.0.0.1")
val port = listener.boundPort()
try {
withContext(Dispatchers.IO) { raw.send(DatagramPacket(bytes, bytes.size, address, port)) }
val carrier = receivedCarriers.receive()
assertEquals(ConcealedCarrier.Kind.DATAGRAM, carrier.kind)
assertContentEquals(bytes, carrier.receive()?.asByteArray())
carrier.send(bytes.asUByteArray())
val packet = DatagramPacket(ByteArray(100), 100)
withContext(Dispatchers.IO) { raw.receive(packet) }
assertContentEquals(bytes, packet.data.copyOf(packet.length))
withContext(Dispatchers.IO) { second.send(DatagramPacket(bytes, bytes.size, address, port)) }
assertNull(withTimeoutOrNull(150) { receivedCarriers.receive() })
// Existing carrier continues to work; an unverified source cannot replace it.
withContext(Dispatchers.IO) { raw.send(DatagramPacket(bytes, bytes.size, address, port)) }
assertContentEquals(bytes, carrier.receive()?.asByteArray())
val waiting = async { runCatching { carrier.receive() } }
listener.close()
waiting.await()
collector.join()
} finally { raw.close(); second.close(); listener.close(); collector.cancelAndJoin() }
}
}
@Test fun udpConnectorHasNoLegacyEnvelopeAndRejectsOversizeLocally() = runBlocking<Unit> {
withTimeout(10_000) {
val raw = DatagramSocket(0, InetAddress.getByName("127.0.0.1")).also { it.soTimeout = 3000 }
val carrier = connectConcealedUdpCarrier(NetworkAddress("127.0.0.1", raw.localPort))
try {
carrier.send(bytes.asUByteArray())
val packet = DatagramPacket(ByteArray(2000), 2000)
withContext(Dispatchers.IO) { raw.receive(packet) }
assertContentEquals(bytes, packet.data.copyOf(packet.length))
withContext(Dispatchers.IO) { raw.send(DatagramPacket(bytes, bytes.size, packet.socketAddress)) }
assertContentEquals(bytes, carrier.receive()?.asByteArray())
assertFailsWith<IllegalArgumentException> { carrier.send(UByteArray(1201)) }
} finally { carrier.close(); raw.close() }
}
}
private suspend fun readExactly(carrier: ConcealedCarrier, n: Int): UByteArray {
var result = ubyteArrayOf()
while (result.size < n) result += carrier.receive() ?: error("Premature EOF")
assertEquals(n, result.size)
return result
}
}
@@ -0,0 +1,58 @@
package net.sergeych.kiloparsec.concealed.elligator
import java.io.File
import java.lang.management.ManagementFactory
import com.sun.management.ThreadMXBean
import kotlinx.coroutines.runBlocking
import net.sergeych.crypto2.initCrypto
import kotlin.test.Test
class ElligatorTimingJvmTest {
@Test fun optInTimingProbe() = runBlocking {
if (System.getenv("KILOPARSEC_TIMING_PROBE") != "1") return@runBlocking
initCrypto()
val count = System.getenv("KILOPARSEC_TIMING_SAMPLES")?.toInt() ?: 256
val warmup = System.getenv("KILOPARSEC_TIMING_WARMUP")?.toInt() ?: 64
if (System.getenv("KILOPARSEC_ALLOCATION_PROBE") == "1") {
measureAllocations(count, warmup)
return@runBlocking
}
val metadata = "JVM ${System.getProperty("java.version")} ${System.getProperty("java.vm.name")}; ${System.getProperty("os.name")} ${System.getProperty("os.arch")}; processors=${Runtime.getRuntime().availableProcessors()}"
val report = ElligatorTimingProbe.run(count, warmup, metadata)
File("build/concealed-timing").mkdirs()
File("build/concealed-timing/jvm.csv").writeText(report)
println(report.substringBefore("raw_group"))
}
private fun measureAllocations(count: Int, warmup: Int) {
require(count in 16..100_000 && warmup in 0..10_000)
val bean = ManagementFactory.getThreadMXBean() as? ThreadMXBean
?: error("This JVM does not expose per-thread allocation counters")
check(bean.isThreadAllocatedMemorySupported)
val wasEnabled = bean.isThreadAllocatedMemoryEnabled
bean.isThreadAllocatedMemoryEnabled = true
val thread = Thread.currentThread().threadId()
val bytes = mutableListOf<Long>()
try {
repeat(count + warmup) { index ->
val before = bean.getThreadAllocatedBytes(thread)
val pair = ExperimentalElligator25519.generateKeyPair()
pair.destroy()
val allocated = bean.getThreadAllocatedBytes(thread) - before
if (index >= warmup) bytes += allocated
}
} finally { bean.isThreadAllocatedMemoryEnabled = wasEnabled }
val sorted = bytes.sorted()
val report = buildString {
appendLine("HotSpot per-current-thread allocated bytes, actual Elligator key generation plus destroy; $count samples after $warmup warmups.")
appendLine("Includes managed temporary arrays/bignums; excludes other threads and native libsodium allocations. Not retained heap, peak RSS, a Native allocation count or a timing-security claim.")
appendLine("mean_bytes,median_bytes,p95_bytes,max_bytes")
appendLine("${bytes.average()},${sorted[sorted.size / 2]},${sorted[((sorted.size - 1) * .95).toInt()]},${sorted.last()}")
appendLine("raw_allocated_bytes")
bytes.forEach { appendLine(it) }
}
File("build/concealed-timing").mkdirs()
File("build/concealed-timing/allocations-jvm.csv").writeText(report)
println(report.substringBefore("raw_allocated_bytes"))
}
}
@@ -0,0 +1,131 @@
package net.sergeych.kiloparsec.concealed.elligator
import kotlinx.coroutines.delay
import kotlinx.coroutines.runBlocking
import java.io.File
import kotlin.math.abs
import kotlin.math.sqrt
import kotlin.random.Random
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertTrue
/** Illustrative response-shaping experiment, NOT a measurement of Elligator or a security proof. */
class TimingShapingProbeTest {
private fun ceilTick(time: Long, tick: Long): Long = ((time + tick - 1) / tick) * tick
@Test fun completionQuantizationLeaksBoundaryCrossingInExactModel() {
// Microseconds; enumerate every phase and convolve the two release buckets with
// every discrete uniform jitter value, without a full Cartesian loop.
val tick = 50_000L
val jitterMaximum = 10_000
fun histogram(duration: Long): LongArray {
val buckets = (0L until tick).groupingBy { phase -> ceilTick(phase + duration, tick).toInt() }.eachCount()
val result = LongArray(110_001)
for ((release, weight) in buckets) for (jitter in 0..jitterMaximum step 1_000) result[release + jitter] += weight.toLong()
return result
}
val fast = histogram(2_000)
val slow = histogram(3_000)
fun mean(histogram: LongArray) = histogram.withIndex().sumOf { it.index.toDouble() * it.value } / histogram.sum()
assertTrue(abs((mean(slow) - mean(fast)) - 1_000.0) < .000001)
// A peer controlling the start phase can amplify the distinction: at phase47.5ms,
// 2ms work ends at49.5ms;3ms ends at50.5ms and releases at100ms, before jitter.
val fastChosen = (0..jitterMaximum step 1_000).map { ceilTick(47_500L + 2_000, tick) + it }
val slowChosen = (0..jitterMaximum step 1_000).map { ceilTick(47_500L + 3_000, tick) + it }
assertTrue(fastChosen.last() < slowChosen.first())
for (phase in 0L until tick) {
val deadline = phase + tick // Selected before work begins, same jitter law for both.
assertEquals(maxOf(deadline, phase + 2_000), maxOf(deadline, phase + 3_000))
}
assertEquals(50_000, (0L until tick).count { phase -> phase + 52_000 > phase + tick })
println("Synthetic exact model: next50ms completion boundary+uniform integer0..10ms jitter: mean2ms=${mean(fast)}us, mean3ms=${mean(slow)}us, gap=1000us. At47.5ms startphase release supports are50..60ms vs100..110ms (disjoint). Start-selected50ms deadline+jitter: identical distributions for both;52ms work overruns budget.")
}
@Test fun syntheticCoroutineReleaseTimingProbe() = runBlocking {
if (System.getenv("KILOPARSEC_TIMING_PROBE") != "1") return@runBlocking
val perClass = System.getenv("KILOPARSEC_SHAPING_SAMPLES")?.toIntOrNull() ?: 100
require(perClass in 10..10_000)
val tick = 50_000_000L
val origin = System.nanoTime()
val random = Random(0x51A9E)
data class Sample(val mode: String, val nanos: Long, val phase: Long, val compute: Long,
val release: Long, val target: Long, val overrun: Boolean)
val samples = ArrayList<Sample>()
// Warm the synthetic loop and coroutine scheduler independently of measured samples.
repeat(20) { spin(2_000_000); delay(1) }
for (mode in listOf("unshaped", "completion_tick_50ms_plus_uniform_0_10ms", "start_deadline_50ms_plus_uniform_0_10ms")) {
val classes = List(perClass * 2) { if (it % 2 == 0) 2_000_000L else 3_000_000L }.shuffled(random)
for (work in classes) {
// Starting immediately after the previous quantized release would restrict
// phases to roughly0..10ms and miss the boundary we intend to investigate.
// Choose an independent phase for every sample; CSV records actual phase,
// including unavoidable coroutine scheduling overshoot.
val phaseTarget = ceilTick(System.nanoTime() - origin, tick) + random.nextLong(tick)
awaitDeadline(origin, phaseTarget)
val jitter = random.nextLong(0, 11) * 1_000_000
val start = System.nanoTime() - origin
val preselected = start + tick + jitter
spin(work)
val done = System.nanoTime() - origin
val target = when (mode) {
"unshaped" -> done
"completion_tick_50ms_plus_uniform_0_10ms" -> ceilTick(done, tick) + jitter
else -> preselected
}
awaitDeadline(origin, target)
val released = System.nanoTime() - origin
samples += Sample(mode, work, start % tick, done - start, released - start,
target - start, done > start + tick)
}
}
val directory = File("build/concealed-timing").apply { mkdirs() }
File(directory, "shaping-samples.csv").printWriter().use { out ->
out.println("mode,class_work_ns,start_phase_ns,compute_ns,release_elapsed_ns,target_elapsed_ns,compute_exceeds_50ms")
samples.forEach { out.println("${it.mode},${it.nanos},${it.phase},${it.compute},${it.release},${it.target},${it.overrun}") }
}
val report = buildString {
appendLine("SYNTHETIC CPU WORK ONLY: this is not Elligator timing evidence or a constant-time/security proof.")
appendLine("JVM=${System.getProperty("java.version")}; OS=${System.getProperty("os.name")}/${System.getProperty("os.arch")}; samples/class/mode=$perClass; seed=0x51A9E")
appendLine("Arbitrary fixed global grid on monotonic clock (not adjustable wall-clock); integer0..10ms jitter; coroutine delay rounds up remaining milliseconds. No claim of precise wall-clock tick release.")
appendLine("Every sample independently selects a uniform0..50ms start phase; CSV records the realized phase after coroutine scheduling.")
for ((mode, group) in samples.groupBy { it.mode }) {
val a = group.filter { it.nanos == 2_000_000L }.map { it.release.toDouble() }
val b = group.filter { it.nanos == 3_000_000L }.map { it.release.toDouble() }
appendLine("$mode: mean2ms=${a.average() / 1e6}ms mean3ms=${b.average() / 1e6}ms absWelchT=${welch(a, b)}")
for ((work, sub) in group.groupBy { it.nanos }) {
val ordered = sub.map { it.release }.sorted()
fun percentile(p: Double) = ordered[((ordered.size - 1) * p).toInt()] / 1e6
appendLine(" work=${work / 1e6}ms median=${percentile(.5)}ms p95=${percentile(.95)}ms p99=${percentile(.99)}ms max=${ordered.last() / 1e6}ms computeBudgetOverruns=${sub.count { it.overrun }}")
}
}
appendLine("Boundary-model evidence is in completionQuantizationLeaksBoundaryCrossingInExactModel. This small live sample is scheduler/load sensitive; a low t score does not establish absence of leakage.")
appendLine("Policy: next50ms boundary AFTER completion + uniform0..10ms jitter versus start-selected50ms budget + same jitter. Start-selected padding suppresses computation-dependent release differences only while work fits its budget. It does not protect local cache/contention observers or hide overruns.")
}
File(directory, "shaping-summary.txt").writeText(report)
println(report)
assertTrue(samples.all { it.release >= it.target })
}
private fun spin(nanos: Long) {
val start = System.nanoTime()
while (System.nanoTime() - start < nanos) Thread.onSpinWait()
}
private suspend fun awaitDeadline(origin: Long, deadline: Long) {
while (true) {
val remaining = deadline - (System.nanoTime() - origin)
if (remaining <= 0) return
delay((remaining + 999_999) / 1_000_000)
}
}
private fun welch(a: List<Double>, b: List<Double>): Double {
fun variance(v: List<Double>): Double {
val mean = v.average()
return v.sumOf { (it - mean) * (it - mean) } / (v.size - 1)
}
val denominator = sqrt(variance(a) / a.size + variance(b) / b.size)
return if (denominator == 0.0) 0.0 else abs(a.average() - b.average()) / denominator
}
}
@@ -0,0 +1,86 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import com.ionspin.kotlin.crypto.scalarmult.ScalarMultiplication
import com.ionspin.kotlin.crypto.util.LibsodiumRandom
import java.io.File
import kotlinx.coroutines.runBlocking
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import kotlin.math.ceil
import kotlin.test.Test
import kotlin.test.assertEquals
/** Explicit CPU-path diagnostic: excludes socket IO, policy callbacks, queueing and scheduling. */
class HandshakeTimingProbeTest {
@Test fun optInTimingProbe() = runBlocking {
if (System.getenv("KILOPARSEC_TIMING_PROBE") != "1") return@runBlocking
initCrypto()
val count = System.getenv("KILOPARSEC_TIMING_SAMPLES")?.toInt() ?: 256
val warmup = System.getenv("KILOPARSEC_TIMING_WARMUP")?.toInt() ?: 64
require(count in 16..100_000 && warmup in 0..10_000)
val secret = LibsodiumRandom.buf(32)
val public = ScalarMultiplication.scalarMultiplicationBase(secret)
val serverKey = SigningSecretKey.new()
val clientKey = SigningSecretKey.new()
val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
val samples = mutableListOf<Pair<String, Long>>()
fun <T> timed(name: String, record: Boolean, action: () -> T): T {
val start = System.nanoTime()
val result = action()
val elapsed = System.nanoTime() - start
if (record) samples += name to elapsed
return result
}
try {
repeat(warmup + count) { round ->
val record = round >= warmup
val name = if (round == 0) "first-in-probe" else "warm"
val start = System.nanoTime()
val client = timed("client.helo.prepare", record) {
ClientHandshake(public, serverKey.verifyingKey, clientKey, ConcealedCarrier.Kind.STREAM, options)
}
var server: ServerHandshake? = null
try {
timed("server.helo.prefix", record) {
assertEquals(client.hello.size, ServerHandshake.helloSize(client.hello.copyOfRange(0, 76), secret))
}
val active = timed("server.helo.respond", record) {
ServerHandshake(secret, serverKey, ConcealedCarrier.Kind.STREAM, options, client.hello)
}
server = active
val ack = timed("client.server.verify-and-ack", record) { client.receiveServerHello(active.response) }
val acceptance = timed("server.ack.verify-and-accept", record) {
active.verifyAck(ack)
active.accept()
}
val session = timed("client.acceptance.verify", record) { client.receiveAcceptance(acceptance) }
val elapsed = System.nanoTime() - start
if (record || round == 0) samples += "handshake.crypto.total.$name" to elapsed
session.sending.destroy(); session.receiving.destroy(); session.token.fill(0u)
active.session.sending.destroy(); active.session.receiving.destroy(); active.session.token.fill(0u)
} finally {
client.destroy(); server?.destroy()
}
}
} finally { secret.fill(0u) }
val report = buildString {
appendLine("JVM actual four-flight crypto paths; no socket latency, application hooks, admission waits or artificial delay. Fresh ephemeral keys each attempt; fixed synthetic server/client identities; no key bytes emitted.")
appendLine("samples=$count warmup=$warmup clock=System.nanoTime padding=0 sequential; first-in-probe is not a process-cold measurement (earlier tests may initialize classes); warm order follows actual handshake.")
appendLine("group,n,mean_ns,median_ns,p95_ns,p99_ns,max_ns")
samples.groupBy { it.first }.forEach { (name, values) ->
val ns = values.map { it.second }.sorted()
fun q(f: Double) = ns[(ceil(f * ns.size).toInt() - 1).coerceIn(0, ns.lastIndex)]
appendLine("$name,${ns.size},${ns.average()},${q(0.5)},${q(0.95)},${q(0.99)},${ns.last()}")
}
appendLine("raw_group,nanos")
samples.forEach { (name, ns) -> appendLine("$name,$ns") }
}
File("build/concealed-timing").mkdirs()
File("build/concealed-timing/handshake-jvm.csv").writeText(report)
println(report.substringBefore("raw_group"))
}
}
@@ -0,0 +1,104 @@
@file:OptIn(ExperimentalUnsignedTypes::class, net.sergeych.kiloparsec.concealed.ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed.handshake
import java.io.File
import kotlinx.coroutines.runBlocking
import net.sergeych.crypto2.DecryptingSecretKey
import net.sergeych.crypto2.SigningSecretKey
import net.sergeych.crypto2.initCrypto
import net.sergeych.kiloparsec.concealed.ConcealedCarrier
import net.sergeych.kiloparsec.concealed.ConcealedOptions
import net.sergeych.kiloparsec.concealed.ConcealedServerIdentity
import kotlin.math.ceil
import kotlin.math.sqrt
import kotlin.random.Random
import kotlin.test.Test
import kotlin.test.assertEquals
import kotlin.test.assertFailsWith
/** Opt-in bounded diagnostic of the actual shared server identity failure wrapper. */
class HelloFailureTimingProbeTest {
@Test fun optInServerHelloFailureProbe() = runBlocking {
if (System.getenv("KILOPARSEC_TIMING_PROBE") != "1") return@runBlocking
initCrypto()
val count = System.getenv("KILOPARSEC_FAILURE_TIMING_SAMPLES")?.toInt() ?: 64
val warmup = 16
require(count in 16..10_000)
val encryption = DecryptingSecretKey.new()
val signing = SigningSecretKey.new()
val clientSigning = SigningSecretKey.new()
val server = ConcealedServerIdentity(encryption, signing, recentDhCapacity = 0)
val options = ConcealedOptions(padding = ConcealedOptions.Padding(0))
val maximum = flightLimit(options, ConcealedCarrier.Kind.STREAM)
val hellos = List(8) {
val client = ClientHandshake(encryption.publicKey.keyBytes, signing.verifyingKey, clientSigning,
ConcealedCarrier.Kind.STREAM, options)
try { client.hello.copyOf() } finally { client.destroy() }
}
data class Sample(val group: String, val nanos: Long)
val samples = mutableListOf<Sample>()
val random = Random(0xFA117)
repeat(warmup + count) { round ->
val hello = hellos[round % hellos.size]
val validPrefix = hello.copyOfRange(0, 76)
// Change authenticated envelope bytes, retaining the same public representative and IV.
val badPrefix = validPrefix.copyOf().also { it[60] = (it[60].toInt() xor 1).toUByte() }
val badBody = hello.copyOf().also { it[it.lastIndex] = (it.last().toInt() xor 1).toUByte() }
val cases = listOf("valid-header", "bad-header", "bad-body").flatMap { kind ->
listOf(false, true).map { enabled -> kind to enabled }
}.shuffled(random)
for ((kind, enabled) in cases) {
val start = System.nanoTime()
server.cryptographicOperation {
when (kind) {
"valid-header" -> assertEquals(hello.size, server.helloAuthentication(validPrefix, maximum, true, enabled) {
ServerHandshake.helloSize(validPrefix, encryption.keyBytes, maximum)
})
"bad-header" -> assertFailsWith<UnauthenticatedFlight> {
server.helloAuthentication(badPrefix, maximum, true, enabled) {
ServerHandshake.helloSize(badPrefix, encryption.keyBytes, maximum)
}
}
else -> assertFailsWith<UnauthenticatedFlight> {
server.helloAuthentication(badBody, maximum, false, enabled) {
ServerHandshake(encryption.keyBytes, signing, ConcealedCarrier.Kind.STREAM, options, badBody)
}
}
}
}
val nanos = System.nanoTime() - start
if (round >= warmup) samples += Sample("$kind.noise-${if (enabled) "on" else "off"}", nanos)
}
}
val groups = samples.groupBy { it.group }
fun variance(values: List<Sample>): Double {
val mean = values.map { it.nanos }.average()
return values.sumOf { (it.nanos - mean) * (it.nanos - mean) } / (values.size - 1)
}
val report = buildString {
appendLine("JVM shared ConcealedServerIdentity cryptographicOperation + helloAuthentication actual code path; no IO, network scheduling, rate limits or configured coroutine delay.")
appendLine("JVM=${System.getProperty("java.version")} OS=${System.getProperty("os.name")}/${System.getProperty("os.arch")}; samples/class=$count warmup=$warmup seed=0xFA117 clock=System.nanoTime.")
appendLine("Eight synthetic valid HELOs reused for paired comparisons; header/body corruptions preserve public representative. Class order randomized each round. Bad body includes ServerHandshake construction work before authentication rejection. Fresh decoy mutation each failed enabled call. No key bytes emitted.")
appendLine("Interpretation: overhead diagnostic only. X25519 is constant-time and dummy Elligator decode sees the same public representative, so mutated key bits are not expected to create substantial deliberate timing noise. No claim of timing-security improvement or constant-time proof.")
appendLine("group,n,mean_ns,median_ns,p95_ns,p99_ns,max_ns")
groups.forEach { (name, values) ->
val ns = values.map { it.nanos }.sorted()
fun q(f: Double) = ns[(ceil(f * ns.size).toInt() - 1).coerceIn(0, ns.lastIndex)]
appendLine("$name,${ns.size},${ns.average()},${q(.5)},${q(.95)},${q(.99)},${ns.last()}")
}
appendLine("case,mean_added_ns,welch_t_on_minus_off")
for (kind in listOf("valid-header", "bad-header", "bad-body")) {
val on = groups.getValue("$kind.noise-on")
val off = groups.getValue("$kind.noise-off")
val difference = on.map { it.nanos }.average() - off.map { it.nanos }.average()
appendLine("$kind,$difference,${difference / sqrt(variance(on) / on.size + variance(off) / off.size)}")
}
appendLine("raw_group,nanos")
samples.forEach { appendLine("${it.group},${it.nanos}") }
}
val directory = File("build/concealed-timing").apply { mkdirs() }
File(directory, "helo-failure-jvm.csv").writeText(report)
println(report.substringBefore("raw_group"))
}
}
@@ -0,0 +1,67 @@
@file:OptIn(ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import kotlin.time.TimeSource
/** Starting rate budgets for previously unseen carrier attempts, before crypto or state allocation. */
@ExperimentalConcealedProtocol
data class ConcealedAdmissionLimits(
val globalPerSecond: Int = 100,
val globalBurst: Int = 200,
val perIpPerSecond: Int = 10,
val perIpBurst: Int = 20,
val maxSourceEntries: Int = 4096,
val sourceIdleMillis: Long = 60_000,
) {
init {
require(globalPerSecond > 0 && globalBurst > 0 && perIpPerSecond > 0 && perIpBurst > 0)
require(maxSourceEntries > 0 && sourceIdleMillis > 0)
}
}
/** Called under its listener's access mutex. Monotonic time, bounded table, no per-source timers. */
internal class NewCarrierAdmission(
private val limits: ConcealedAdmissionLimits,
private val nowMillis: () -> Double = monotonicClock(),
) {
private data class Bucket(var tokens: Double, var updatedAt: Double, var lastUsedAt: Double)
private val global = Bucket(limits.globalBurst.toDouble(), nowMillis(), nowMillis())
private val sources = mutableMapOf<String, Bucket>()
private var lastSweep = global.updatedAt
fun admit(ip: String): Boolean {
val now = nowMillis()
if (now - lastSweep >= minOf(1000.0, limits.sourceIdleMillis.toDouble())) {
val iterator = sources.iterator()
while (iterator.hasNext()) {
if (now - iterator.next().value.lastUsedAt >= limits.sourceIdleMillis) iterator.remove()
}
lastSweep = now
}
refill(global, now, limits.globalPerSecond, limits.globalBurst)
if (global.tokens < 1.0) return false
val source = sources[ip] ?: run {
if (sources.size >= limits.maxSourceEntries) return false
Bucket(limits.perIpBurst.toDouble(), now, now).also { sources[ip] = it }
}
refill(source, now, limits.perIpPerSecond, limits.perIpBurst)
source.lastUsedAt = now
if (source.tokens < 1.0) return false
global.tokens -= 1.0
source.tokens -= 1.0
return true
}
private fun refill(bucket: Bucket, now: Double, rate: Int, burst: Int) {
bucket.tokens = minOf(burst.toDouble(), bucket.tokens + maxOf(0.0, now - bucket.updatedAt) * rate / 1000.0)
bucket.updatedAt = now
}
companion object {
private fun monotonicClock(): () -> Double {
val origin = TimeSource.Monotonic.markNow()
return { origin.elapsedNow().inWholeNanoseconds / 1_000_000.0 }
}
}
}
@@ -0,0 +1,124 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import io.ktor.network.selector.SelectorManager
import io.ktor.network.sockets.*
import io.ktor.utils.io.*
import kotlinx.coroutines.*
import kotlinx.coroutines.flow.flow
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.kiloparsec.adapter.NetworkAddress
/** Raw TCP bytes: no legacy varint envelope, ping bytes, or additional scheduling. */
@ExperimentalConcealedProtocol
suspend fun connectConcealedTcpCarrier(address: NetworkAddress): ConcealedCarrier = withContext(Dispatchers.IO) {
val selector = SelectorManager(Dispatchers.IO)
try {
RawTcpCarrier(aSocket(selector).tcp().connect(address.host, address.port)) { selector.close() }
} catch (e: Throwable) {
selector.close()
throw e
}
}
private class RawTcpCarrier(
private val socket: Socket,
private val onClose: suspend () -> Unit = {},
) : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.STREAM
private val input = socket.openReadChannel()
private val output = socket.openWriteChannel()
private val closeAccess = Mutex()
private val writeAccess = Mutex()
private var closed = false
override suspend fun receive(): UByteArray? {
val buffer = ByteArray(8192)
val n = input.readAvailable(buffer, 0, buffer.size)
return if (n < 0) null else buffer.copyOf(n).asUByteArray()
}
override suspend fun send(bytes: UByteArray) = writeAccess.withLock {
check(!socket.isClosed) { "Carrier is closed" }
output.writeFully(bytes.asByteArray(), 0, bytes.size)
output.flush()
}
override suspend fun close(): Unit = withContext(NonCancellable) {
closeAccess.withLock {
if (!closed) {
closed = true
runCatching { input.cancel() }
runCatching { socket.close() }
runCatching { onClose() }
}
}
}
}
/** Managed, single-collector raw TCP listener. Collectors launch connection handling separately. */
@ExperimentalConcealedProtocol
class ConcealedTcpListener(
port: Int = 0, localInterface: String = "0.0.0.0",
private val maxCarriers: Int = 1024, private val maxCarriersPerIp: Int = 8,
admissionLimits: ConcealedAdmissionLimits = ConcealedAdmissionLimits(),
) {
init { require(maxCarriers > 0 && maxCarriersPerIp > 0) }
private val selector = SelectorManager(Dispatchers.IO)
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private val socket = scope.async { aSocket(selector).tcp().bind(localInterface, port) }
private val access = Mutex()
private val connections = mutableMapOf<ConcealedCarrier, String>()
private var closed = false
private var collected = false
private val admission = NewCarrierAdmission(admissionLimits)
suspend fun boundPort(): Int = (socket.await().localAddress as InetSocketAddress).port
val carriers: Flow<ConcealedCarrier> = flow {
access.withLock { check(!collected) { "Collect this listener once" }; collected = true }
try {
val server = socket.await()
while (currentCoroutineContext().isActive) {
val accepted = try { server.accept() }
catch (e: Exception) {
currentCoroutineContext().ensureActive()
if (access.withLock { closed }) break else throw e
}
val carrier = try { access.withLock {
val source = (accepted.remoteAddress as InetSocketAddress).hostname
if (closed || connections.size >= maxCarriers ||
connections.values.count { it == source } >= maxCarriersPerIp || !admission.admit(source)) {
accepted.close(); null
}
else {
lateinit var result: ConcealedCarrier
result = RawTcpCarrier(accepted) { access.withLock { connections.remove(result) }; Unit }
connections[result] = source
result
}
} } catch (e: Throwable) {
// Cancellation between accept and registration must not leak this socket.
runCatching { accepted.close() }
throw e
}
if (carrier != null) emit(carrier)
}
} finally { close() }
}
suspend fun close(): Unit = withContext(NonCancellable) {
val pending = access.withLock {
if (closed) return@withContext
closed = true
connections.keys.toList().also { connections.clear() }
}
runCatching { socket.await().close() }
pending.forEach { it.close() }
scope.cancel()
runCatching { selector.close() }
}
}
@@ -0,0 +1,167 @@
@file:OptIn(ExperimentalUnsignedTypes::class, ExperimentalConcealedProtocol::class)
package net.sergeych.kiloparsec.concealed
import io.ktor.network.selector.SelectorManager
import io.ktor.network.sockets.*
import io.ktor.utils.io.core.ByteReadPacket
import io.ktor.utils.io.core.readBytes
import kotlinx.coroutines.*
import kotlinx.coroutines.channels.Channel
import kotlinx.coroutines.flow.flow
import kotlinx.coroutines.flow.Flow
import kotlinx.coroutines.sync.Mutex
import kotlinx.coroutines.sync.withLock
import net.sergeych.kiloparsec.adapter.NetworkAddress
/** Resource bounds for raw demultiplexing, including established connections. */
@ExperimentalConcealedProtocol
data class ConcealedUdpListenerLimits(
val maxCarriers: Int = 1024,
val maxCarriersPerIp: Int = 8,
val bufferedDatagramsPerCarrier: Int = 32,
val maxDatagramBytes: Int = 1200,
) {
init {
require(maxCarriers > 0 && maxCarriersPerIp > 0 && bufferedDatagramsPerCarrier > 0)
require(maxDatagramBytes in 512..65507)
}
}
/** Connected socket preserves datagrams and lets the OS reject unrelated sources. */
@ExperimentalConcealedProtocol
suspend fun connectConcealedUdpCarrier(
address: NetworkAddress, maxDatagramBytes: Int = 1200,
): ConcealedCarrier = withContext(Dispatchers.IO) {
require(maxDatagramBytes in 512..65507)
val selector = SelectorManager(Dispatchers.IO)
try {
val remote = InetSocketAddress(address.host, address.port)
val socket = aSocket(selector).udp().connect(remote)
object : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.DATAGRAM
private val closeAccess = Mutex()
private val sendAccess = Mutex()
private var closed = false
override suspend fun receive(): UByteArray? {
while (currentCoroutineContext().isActive) {
val packet = try { socket.receive().packet }
catch (e: Exception) {
currentCoroutineContext().ensureActive()
if (closeAccess.withLock { closed }) return null else throw e
}
val bytes = packet.readBytes()
if (bytes.size > maxDatagramBytes) continue
return bytes.asUByteArray()
}
currentCoroutineContext().ensureActive()
return null
}
override suspend fun send(bytes: UByteArray) = sendAccess.withLock {
require(bytes.size <= maxDatagramBytes) { "Datagram exceeds configured budget" }
socket.send(Datagram(ByteReadPacket(bytes.asByteArray()), remote))
}
override suspend fun close(): Unit = withContext(NonCancellable) {
closeAccess.withLock {
if (!closed) {
closed = true
runCatching { socket.close() }
runCatching { selector.close() }
}
}
}
}
} catch (e: Throwable) {
selector.close()
throw e
}
}
/**
* Raw UDP listener with bounded per-source carriers and drop-on-full input queues.
* New addresses never replace an existing carrier. The handshake supervisor must close failed
* or timed-out attempts, apply crypto concurrency/rate budgets, and launch handlers independently
* while collecting this flow. This adapter does not infer authorization from incoming bytes.
*/
@ExperimentalConcealedProtocol
class ConcealedUdpListener(
port: Int = 0, localInterface: String = "0.0.0.0",
private val limits: ConcealedUdpListenerLimits = ConcealedUdpListenerLimits(),
admissionLimits: ConcealedAdmissionLimits = ConcealedAdmissionLimits(),
) {
private val selector = SelectorManager(Dispatchers.IO)
private val scope = CoroutineScope(SupervisorJob() + Dispatchers.IO)
private val socket = scope.async { aSocket(selector).udp().bind(InetSocketAddress(localInterface, port)) }
private val access = Mutex()
private val sendAccess = Mutex()
private val sessions = mutableMapOf<SocketAddress, Peer>()
private var closed = false
private var collected = false
private val admission = NewCarrierAdmission(admissionLimits)
suspend fun boundPort(): Int = (socket.await().localAddress as InetSocketAddress).port
private inner class Peer(val address: InetSocketAddress) : ConcealedCarrier {
override val kind = ConcealedCarrier.Kind.DATAGRAM
val incoming = Channel<UByteArray>(limits.bufferedDatagramsPerCarrier)
private var stopped = false // guarded by listener access
override suspend fun receive(): UByteArray? = incoming.receiveCatching().getOrNull()
override suspend fun send(bytes: UByteArray) {
require(bytes.size <= limits.maxDatagramBytes) { "Datagram exceeds configured budget" }
check(access.withLock { !closed && !stopped }) { "Carrier is closed" }
sendAccess.withLock { socket.await().send(Datagram(ByteReadPacket(bytes.asByteArray()), address)) }
}
override suspend fun close(): Unit = withContext(NonCancellable) {
access.withLock {
if (!stopped) {
stopped = true
if (sessions[address] === this@Peer) sessions.remove(address)
incoming.cancel()
}
}
}
}
val carriers: Flow<ConcealedCarrier> = flow {
access.withLock { check(!collected) { "Collect this listener once" }; collected = true }
try {
val server = socket.await()
while (currentCoroutineContext().isActive) {
val datagram = try { server.receive() }
catch (e: Exception) {
currentCoroutineContext().ensureActive()
if (access.withLock { closed }) break else throw e
}
val packet = datagram.packet
val bytes = packet.readBytes().asUByteArray()
if (bytes.size > limits.maxDatagramBytes) continue
val address = datagram.address as? InetSocketAddress ?: continue
var created = false
val peer = access.withLock {
if (closed) null else sessions[address] ?: run {
val count = sessions.values.count { it.address.hostname == address.hostname }
if (sessions.size >= limits.maxCarriers || count >= limits.maxCarriersPerIp ||
!admission.admit(address.hostname)) null
else Peer(address).also { sessions[address] = it; created = true }
}
}
if (peer != null) {
peer.incoming.trySend(bytes) // overflow drops newest; no global socket stall
if (created) emit(peer)
}
}
} finally { close() }
}
suspend fun close(): Unit = withContext(NonCancellable) {
val pending = access.withLock {
if (closed) return@withContext
closed = true
sessions.values.toList().also { sessions.clear() }
}
runCatching { socket.await().close() }
pending.forEach { it.close() }
scope.cancel()
runCatching { selector.close() }
}
}
@@ -0,0 +1,31 @@
@file:OptIn(kotlinx.cinterop.ExperimentalForeignApi::class)
package net.sergeych.kiloparsec.concealed.elligator
import kotlinx.cinterop.toKString
import kotlinx.coroutines.runBlocking
import net.sergeych.crypto2.initCrypto
import platform.posix.fclose
import platform.posix.fopen
import platform.posix.fputs
import platform.posix.getenv
import platform.posix.mkdir
import kotlin.test.Test
class ElligatorTimingNativeTest {
@Test fun optInTimingProbe() = runBlocking {
if (getenv("KILOPARSEC_TIMING_PROBE")?.toKString() != "1") return@runBlocking
initCrypto()
val count = getenv("KILOPARSEC_TIMING_SAMPLES")?.toKString()?.toInt() ?: 256
val warmup = getenv("KILOPARSEC_TIMING_WARMUP")?.toKString()?.toInt() ?: 64
val flavor = getenv("KILOPARSEC_TIMING_FLAVOR")?.toKString() ?: "debug"
require(flavor in listOf("debug", "release"))
val report = ElligatorTimingProbe.run(count, warmup, "Kotlin/Native linuxX64 $flavor test binary")
mkdir("build", 493u)
mkdir("build/concealed-timing", 493u)
val name = if (flavor == "debug") "linuxX64.csv" else "linuxX64-release.csv"
val output = checkNotNull(fopen("build/concealed-timing/$name", "w"))
try { check(fputs(report, output) >= 0) } finally { fclose(output) }
println(report.substringBefore("raw_group"))
}
}
+54
View File
@@ -0,0 +1,54 @@
{
"serverSecret": "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f",
"serverPublic": "8f40c5adb68f25624ae5b214ea767a6ec94d829d3d7b5e1ad1ba6f3e2138285f",
"clientDhSecret": "202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f",
"clientDhPublic": "358072d6365880d1aeea329adf9121383851ed21a28e3b75e965d0d2cd166254",
"serverDhSecret": "404142434445464748494a4b4c4d4e4f505152535455565758595a5b5c5d5e5f",
"serverDhPublic": "79a631eede1bf9c98f12032cdeadd0e7a079398fc786b88cc846ec89af85a51a",
"serverSigningSeed": "606162636465666768696a6b6c6d6e6f707172737475767778797a7b7c7d7e7f",
"serverIdentity": "174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd5",
"clientSigningSeed": "808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f",
"clientIdentity": "cd14b37f956e953194ff7fb73b3d81dcc561d61a7538094b7c3e1a643ee5f3aa",
"representative": "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebf",
"heloIv": "c0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d7",
"bootstrapShared": "900702f8b7a7dd869cf98f17dce31515a39e8043636352e8bf7be99043b83101",
"bootstrapRoot": "f0cce9735754e6985895bbf2ad73a16a1ad5732399df991a598f17e71f3b2d04",
"heloCore": "0101358072d6365880d1aeea329adf9121383851ed21a28e3b75e965d0d2cd166254174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd500100000000000000000",
"f1": "a0a1a2a3a4a5a6a7a8a9aaabacadaeafb0b1b2b3b4b5b6b7b8b9babbbcbdbebfc0c1c2c3c4c5c6c7c8c9cacbcccdcecfd0d1d2d3d4d5d6d77e7c60745d6ff2d29cb31ea1ee07d57dcbf42a3b1ac4341763a6f7baaa5766561e2fd98b7e4a69846548854a72be3b036eac464732b81a97afd4ef73c75f0a45703fd77c5ee8d29188ea452d9125e0bd8b798c212fccb1a65b4b617c1ebffd3c10256bc31f2949c6095f8cbc241d43b12f1591d01df46548",
"h1": "602f30edcbbd47efaeaf6c79e30d0a418ac5dd0f18f03148dc55d6f1115ac6c1",
"replyIv": "000102030405060708090a0b0c0d0e0f1011121314151617",
"replyRoot": "7b747b6d95ad2bbb29d75ffd307291a1fbe9ac30e29fd8723ad859c7edd9cd50",
"serverCore": "010179a631eede1bf9c98f12032cdeadd0e7a079398fc786b88cc846ec89af85a51a56a735db2bdb65b5240a7c7265917a909c1515ca682dcce740c9f00ac91eabcd174553b456dddfc6908ecab1c101fe6ab21e2baa0617795b7d43a63482993fd500100000000000000000",
"serverSignature": "c9449226a851142979083bb1c633005c6323be2c868b2c188db04e63c9dd8952f1bfb549b4a43fa485cf456fe139206f92adf8a738e1ca4d95f03b0378d22906",
"f2": "000102030405060708090a0b0c0d0e0f10111213141516179e23f0972e103c06d5b5a374b83490d3746d51aeb4efb24829822dc8a6826e417734a39ad43437e12d7e379cf633d2397a9ede00b477308a59f262f12c0804f79b7b00b86f04426c7349d53d4935601dad6eb81a95e0bbdab3d73607a6187e2bd18a83d90c93858f17dee1753cfb56395c505a004fc1cbcc2ca6c3820c81b3128c8f69138e7435dfba773b8a05cfa7a328d4bc76dd0db0aba878718c48798ee60a95cb46d24d87003a688e0eb3761871f33621b245f70d1152358cbc41e7b073de23d41579f81883725035a46248c1c17a59644e43ac9e6a",
"h2": "ccb3ed241a73bd60911c36356a70651e8ab884ed5958514122f35d382d0d2533",
"sessionShared": "04c304fb1ca83cee75e206344231f33797e07d9929db670994b7c6fbeb1dc255",
"s2c": "46d6c3a94b432fab260e6252e53b488c6ce8b5dca9d17499facac2e402f1ccb1",
"c2s": "2839576c1187d37c4e4d330ce4331c305c9ebf1ac599110fd1e9c44c1f64a5f3",
"token": "b6e3c9512701a9edea03d60c5f50ac894bf652c776e3636f642a319357148481",
"finishedC2s": "4e6ed04a46d1f01c64873c0050afb167b110de66ae8ab97344e79d0c527bd8ad",
"finishedS2c": "f6de33390f63d704bf2fa82dac7aa1e21a08da48351326047d68c6dd6c9d280e",
"clientSignature": "dc405b92ddf7499d47f3a32e837bed640d4565632e4f332ccbc7b6159a0afd1e1272bd36531a6b5d8a40c60e4a307b8a8e8aca62ba4d635ba278b19e5c448004",
"ackCore": "03ccb3ed241a73bd60911c36356a70651e8ab884ed5958514122f35d382d0d2533cd14b37f956e953194ff7fb73b3d81dcc561d61a7538094b7c3e1a643ee5f3aadc405b92ddf7499d47f3a32e837bed640d4565632e4f332ccbc7b6159a0afd1e1272bd36531a6b5d8a40c60e4a307b8a8e8aca62ba4d635ba278b19e5c448004",
"ackIv": "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f",
"f3": "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f13d609bbf798a1056e5bca20eb89872e2160808864b58cf718c89dabe170b453ccfff09a287502ab1167011f79bbcdabdd673a8e26b22f4f7e7d91ec802dc1b8917820d93a24faaa86abfd0c04ec6c670a5938ac8e003aa7378e6a995b3039e7d5d1a0dd233c8589feb771ee1a6df8d8e43a6ace5ce146ca440eaaa0c3ff0cfe190be2dd60a9c129ae4cb5e3802fd306e1086a180605c59c01e1a17ffb943e517a6516f409159b1b6a9ce95079",
"h3": "4f21ca3091f37bf39d78fea6cb5ea6dca91f0fd78442beb6874fc6a9097be5c0",
"acceptCore": "044f21ca3091f37bf39d78fea6cb5ea6dca91f0fd78442beb6874fc6a9097be5c0",
"acceptIv": "303132333435363738393a3b3c3d3e3f4041424344454647",
"f4": "303132333435363738393a3b3c3d3e3f40414243444546479c1359d7f05d126e96c469773c8a586c711742c1547137a75fed8f3e29c2bb51cf8e86698471e4e4ab58e325efe7c451c6183ae167602430d9a9773baec43dccd6d4d4c067b012838fabaf7624",
"c2sStreamKey": "3849e6ec55adeb429a2f1152fe1d6499f3c331d0c20f961ffb07d37d0a1f6ba6",
"c2sStreamSalt": "124c709383f7fec219af21b040cafa2c28ee23b58bdbd36e",
"c2sMacKey": "098146c2016cc5d4799e804784c424746b75b4f0dfe5ad6772147f48424cfafe",
"c2sUdpKey": "e39c67192f6668ff5e3fe2b3155d161c30327d855d1ad7584337ee0eed3d2d85",
"s2cStreamKey": "7d859e27ebf54f5117dc173919f4952ca4a79d449b39ca4cb66dc982ad6df43c",
"s2cStreamSalt": "98f157f145f68670eafcc2a1e114ae269d5efe1fbc89d806",
"s2cMacKey": "902cce94c7e7335027c3ed5ac695e6ba9ebfef8a2d8dd94bd7bf522b2f78a185",
"s2cUdpKey": "fb470d7c6dd2980c070303660e7d11726a3b92fab057b5d6a2f63a75fc1b9453",
"object0": "000102",
"udp0": "000102030405060708090a0b0c0d0e0f101112131415161752519f6c14b9af07c4ff6e4506bded91385f473d499ae7a1bd77ff90835751",
"object1": "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f303132333435363738393a3b3c3d3e3f404142434445",
"udp1": "18191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f27e785c70cb414797285f7ece74d227f7c8773bafdc1fc59c19631825e48b1352adee3f0c8a9a61c1d49c40f6207fde77ccc279982109f22b1b33ce02f71ffe3565d6c5aa656d58caa8aded51d35c6cef5aefd5b8c08b7c892901bfbfad7c503f7ec",
"object2": "010000",
"udp2": "303132333435363738393a3b3c3d3e3f4041424344454647781578e7e4cdfc68bec534773c724439f04787b1f981d09987320ac7637f21",
"tcp": "7076dc0a75456996a85759de0c9e1ddc7f19e4f42f8d21b9c67909f4e106a0388494a18df501d78b8e51bf3affa6cd10bff8307afc5b4c45494a0e89d2d11a823a7257f7d52c0379d3333e33753933630e270d5fa61f11106d80095feff564231280114d47d12d9af30825d86bb1bbf6339c0047f65976ec0c370230fab18a2f7d79ea93d848b069f0a99b42dc6bf2b6ef96f498"
}
+111
View File
@@ -0,0 +1,111 @@
#!/usr/bin/env python3
"""Independent suite-1 composition oracle. Synthetic secrets ONLY; stdlib + system libsodium.
No Kotlin code or output is imported. Regenerate with: python3 tools/concealed-vectors/reference.py
"""
import ctypes as c
import ctypes.util
import hashlib
import json
import struct
from pathlib import Path
s = c.CDLL(ctypes.util.find_library('sodium'))
s.sodium_init()
def call(name, n, *args):
out = c.create_string_buffer(n)
assert getattr(s, name)(out, *args) == 0, name
return out.raw
def h(x): return hashlib.blake2b(x,digest_size=32).digest()
def u32(x): return struct.pack('>I',x)
def u64(x): return struct.pack('>Q',x)
def v(x): return u32(len(x))+x
def label(x): return v(x.encode())
def kdf(n,k,l,ctx): return hashlib.blake2b(label('kiloparsec/concealed/1')+label(l)+v(ctx),key=k,digest_size=n).digest()
def box(m,n,k): return call('crypto_secretbox_easy',len(m)+16,m,c.c_ulonglong(len(m)),n,k)
def envelope(k,ctx,m):
length=u32(len(m))
return b''.join(box(part,kdf(24,k,'flight/'+name+'-iv',ctx),kdf(32,k,'flight/'+name+'-key',ctx)) for name,part in [('header',length),('body',length+m)])
def signkey(seed):
pk=c.create_string_buffer(32); sk=c.create_string_buffer(64)
assert s.crypto_sign_seed_keypair(pk,sk,seed)==0
return pk.raw,sk.raw
def sign(m,sk):
out=c.create_string_buffer(64); size=c.c_ulonglong()
assert s.crypto_sign_detached(out,c.byref(size),m,c.c_ulonglong(len(m)),sk)==0
return out.raw
def decode(r):
p=2**255-19; A=486662; r=int.from_bytes(r,'little')&((1<<254)-1)
w=-A*pow((1+2*r*r)%p,-1,p)%p
square=pow((w**3+A*w*w+w)%p,(p-1)//2,p) in (0,1)
return (w if square else (-w-A)%p).to_bytes(32,'little')
# Original ChaCha20 implementation: 64-bit counter and 64-bit nonce.
def chacha(key,nonce,data):
def rot(x,n): return ((x<<n)|(x>>(32-n)))&0xffffffff
def quarter(x,a,b,c_,d):
x[a]=(x[a]+x[b])&0xffffffff; x[d]=rot(x[d]^x[a],16)
x[c_]=(x[c_]+x[d])&0xffffffff; x[b]=rot(x[b]^x[c_],12)
x[a]=(x[a]+x[b])&0xffffffff; x[d]=rot(x[d]^x[a],8)
x[c_]=(x[c_]+x[d])&0xffffffff; x[b]=rot(x[b]^x[c_],7)
out=bytearray()
for counter in range((len(data)+63)//64):
state=list(struct.unpack('<4I',b'expand 32-byte k'))+list(struct.unpack('<8I',key))+[counter&0xffffffff,counter>>32]+list(struct.unpack('<2I',nonce))
x=state[:]
for _ in range(10):
for q in [(0,4,8,12),(1,5,9,13),(2,6,10,14),(3,7,11,15),(0,5,10,15),(1,6,11,12),(2,7,8,13),(3,4,9,14)]: quarter(x,*q)
out.extend(struct.pack('<16I',*((a+b)&0xffffffff for a,b in zip(x,state))))
return bytes(a^b for a,b in zip(data,out))
def main():
f={}
def put(name,value): f[name]=value.hex(); return value
e=put('serverSecret',bytes(range(32)))
E=put('serverPublic',call('crypto_scalarmult_base',32,e))
cs=put('clientDhSecret',bytes(range(32,64))); C=put('clientDhPublic',call('crypto_scalarmult_base',32,cs))
ss=put('serverDhSecret',bytes(range(64,96))); S=put('serverDhPublic',call('crypto_scalarmult_base',32,ss))
vs,sks=signkey(put('serverSigningSeed',bytes(range(96,128)))); put('serverIdentity',vs)
vc,skc=signkey(put('clientSigningSeed',bytes(range(128,160)))); put('clientIdentity',vc)
R=put('representative',bytes(range(160,192))); U=put('heloIv',bytes(range(192,216)))
shared=put('bootstrapShared',call('crypto_scalarmult',32,e,decode(R)))
context=R+U+E; root=put('bootstrapRoot',kdf(32,shared,'bootstrap',context))
bounds=u32(1048576)+struct.pack('>H',0)+u32(0)
core=put('heloCore',bytes([1,1])+C+vs+bounds)
F1=put('f1',R+U+envelope(root,h(context),v(core)))
h1=put('h1',h(label('helo')+v(F1)))
I=put('replyIv',bytes(range(24))); replyroot=put('replyRoot',kdf(32,C,'server-reply',h1+E+vs))
core=put('serverCore',bytes([1,1])+S+h(label('helo-decrypted')+C)+vs+bounds)
sig=put('serverSignature',sign(label('kiloparsec/concealed/1/server-auth')+h1+v(core),sks))
F2=put('f2',I+envelope(replyroot,h(label('server-flight')+h1+I),v(core)+sig))
h2=put('h2',h(label('handshake')+v(F1)+v(F2)))
# Independent crypto_kx formula, not a crypto_kx API call.
shared=put('sessionShared',call('crypto_scalarmult',32,cs,S))
kx=hashlib.blake2b(shared+C+S,digest_size=64).digest()
s2c=put('s2c',kx[:32]); c2s=put('c2s',kx[32:])
token=put('token',kdf(32,c2s+s2c,'connection-token',h2))
finishc=put('finishedC2s',kdf(32,c2s,'finished/c2s',h2)); finishs=put('finishedS2c',kdf(32,s2c,'finished/s2c',h2))
sig=put('clientSignature',sign(label('kiloparsec/concealed/1/client-auth')+token+h2+vc,skc))
core=put('ackCore',bytes([3])+h2+vc+sig); A=put('ackIv',bytes(range(24,48)))
F3=put('f3',A+envelope(finishc,h(label('ack')+h2+A),v(core)))
h3=put('h3',h(label('authenticated-handshake')+h2+v(F3)))
core=put('acceptCore',bytes([4])+h3); J=put('acceptIv',bytes(range(48,72)))
put('f4',J+envelope(finishs,h(label('accept')+h2+J),v(core)))
for name,key in [('c2s',c2s),('s2c',s2c)]:
for suffix,label_,n in [('StreamKey','stream',32),('StreamSalt','salt',24),('MacKey','mac',32),('UdpKey','udp',32)]: put(name+suffix,kdf(n,key,'traffic/'+label_+'/'+name,h3))
get=lambda name: bytes.fromhex(f[name])
streamkey=hashlib.blake2b(b'kiloparsec/concealed/stream-key/v1'+get('c2sStreamSalt'),key=get('c2sStreamKey'),digest_size=32).digest()
streamnonce=hashlib.blake2b(b'kiloparsec/concealed/stream-nonce/v1'+get('c2sStreamSalt'),key=get('c2sStreamKey'),digest_size=16).digest()[:8]
objects=[bytes([0,1,2]),bytes(range(70)),bytes([1,0,0])]; plain=b''
for q,obj in enumerate(objects):
put('object'+str(q),obj); body=v(obj); length=u32(len(body))
tag=hashlib.blake2b(b'kiloparsec/concealed/record-mac/v1'+u64(q)+length+body,key=get('c2sMacKey'),digest_size=16).digest()
plain+=length+body+tag
iv=bytes((i+q*24)%256 for i in range(24))
put('udp'+str(q),iv+box(u64(q)+body,iv,get('c2sUdpKey')))
put('tcp',chacha(streamkey,streamnonce,plain))
Path('tools/concealed-vectors/fixtures.json').write_text(json.dumps(f,indent=2)+'\n')
kotlin='''@file:OptIn(ExperimentalUnsignedTypes::class)\npackage net.sergeych.kiloparsec.concealed.handshake\n\n// Generated by tools/concealed-vectors/reference.py; synthetic secrets ONLY.\ninternal object IndependentWireFixtures {\n fun bytes(name: String): UByteArray = values.getValue(name).chunked(2).map { it.toUByte(16) }.toUByteArray()\n private val values = mapOf(\n'''+''.join(' "'+k+'" to "'+value+'",\n' for k,value in f.items())+' )\n}\n'
Path('src/commonTest/kotlin/net/sergeych/kiloparsec/concealed/handshake/IndependentWireFixtures.kt').write_text(kotlin)
print('Generated',len(f),'synthetic fixture fields')
if __name__=='__main__': main()
+3 -3
View File
@@ -1,7 +1,7 @@
plugins { plugins {
kotlin("jvm") version "2.2.21" apply false kotlin("jvm") version "2.4.0" apply false
kotlin("multiplatform") version "2.2.21" apply false kotlin("multiplatform") version "2.4.0" apply false
kotlin("plugin.serialization") version "2.2.21" apply false kotlin("plugin.serialization") version "2.4.0" apply false
} }
allprojects { allprojects {
repositories { repositories {