kiloparsec/README.md

# Kiloparsec

__Recommended version is `0.6.8`: to keep the code compatible with current and further versions we
ask to upgrade to `0.4.2` at least.__ Starting from this version some package names are changed for
better clarity and fast UDP endpoints are added.

The new generation of __PARanoid SECurity__ protocol, advanced, faster, more secure. It also allows connecting any "
block device" transport to the same local interface. Out if the box it
provides the following transports:

| name              | JVM    | JS | wasmJS | native |
|-------------------|--------|----|--------|--------|
| TCP/IP server     | ✓      |    |        | 0.2.6+ |
| TCP/IP client     | ✓      |    |        | 0.2.6+ |
| UDP server        | 0.3.2+ |    |        | 0.3.2+ |
| UDP client        | 0.3.2+ |    |        | 0.3.2+ |
| Websockets server | ✓      |    |        |        |
| Websockets client | ✓      | ✓  | ✓      | ✓      |

### Note on version compatibility

We recommend using `0.6.12`

Since version 0.6.9 websocket protocol supports both text and binary frames; old clients are backward compatible with
mew servers, but new clients only can work with older servers only in default binary frame mode. Upgrade also your
servers to get better websocket compatibility[^1].

Version 0.5.1 could be backward incompatible due to the upgrade of the crypto2.

Protocols >= 0.3.0 are not binary compatible with the previous version due to a more compact binary
format. The format from 0.3.0 onwards is supposed to keep compatible.

#### ID calculation algorithm is changed since 0.4.1

We recommend to upgrade to 0.4+ ASAP as public/shared key id derivation method was changed for even higher security.

### Supported native targets

- iosArm64, iosX64
- macosArm64, macosArm64
- linuxArm64, linuxX64

### Non-native targets

- JS (browser and Node.js)
- JVM (android, macOS, windows, linux, everywhere where JRE is installed)

## TCP/IP and UDP transports

These are the fastest based on async socket implementation of a ktor client. They work everywhere but JS target as
there are currently no widely adopted sockets for browser JavaScript.

While UDP is faster than TCP/IP, it is less reliable, especially with commands and return values that serialize to more
than 240 bytes approx, and has no retransmission facilities (use TCP!). UDP, though, shines when all you need is
to [push](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-remote-interface/push.html) with
little or no data in it.

## Websockets server

While it is much slower than TCP or UDP, it is still faster than any http-based API; it uses binary frames based on
the Ktor server framework to easily integrate with web services. We recommend using it instead of a classic HTTP API as
it beats it in terms of speed and server load even with HTTP/2.

We recommend to create the `KiloInterface<S>` instance and connect it to the websockets and tcp servers in real
applications to get easy access from anywhere.

## Websocket client

It is slower than TCP or UDP, but it works on literally all platforms. See the sample below.

# Usage

The library should be used as a maven dependency, not as source.

## Adding dependency

### Declare maven repository:

Add the private repository to your `build.gradle.kts`, like:

```kotlin
repositories {
    maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
}
```

### Add dependency block

It could be, depending on your project structure, something like:

```kotlin
val commonMain by getting {
    dependencies {
        api("net.sergeych:kiloparsec:0.6.8")
    }
}
```

## Create a shared interface for your server and the client

It could be a multiplatform library that exports it or just a shared or copied source file declaring structures
and functions available, like:

```kotlin
// Api.kt

@Serializable
class FooArgs(val text: String, val number: Int = 42)

// Server-side interface
val cmdSetFoo by command<FooArgs, Unit>()
val cmdGetFoo by command<Unit, FooArgs>()
val cmdPing by command<String, String>()
val cmdCheckConnected by command<Unit, Boolean>()

// client-side interface (called from the server)
val cmdPushClient by command<String, Unit>()
```

## Call it from the client:

Remember, we need to implement client interface `cmdPushClient` in our example, so we need to provide
local interface too:

```kotlin
// Unit: no session on the client:
val client = websocketClient<Unit>("wss://your.host.com/kp") {
    // This is server-callable function we export:
    on(cmdPushClient) {
        "server push: $it"
    }
}

// If we want to collect connected state changes (this is optional)
launch {
    client.connectedStateFlow.collect {
        if (it)
            println("I am connected")
        else
            println("trying to connect...")
    }
}

// now we can call server's functions
client.call(cmdSetFoo, FooArgs("bar", 117))
assertEquals(FooArgs("bar", 117), client.call(cmdGetFoo))

```

## Create a ktor-based server

Normally the server side needs some session. It is convenient and avoids sending repeating data on each request speeding up
the protocol. With KILOPARSEC, it is a rather basic operation:

~~~kotlin
// Our session just keeps Foo for cmd{Get|Set}Foo:
data class Session(var fooState: FooArgs? = null)

// Let's now provide interface we export, it will be used on each connection automatically:

// Note server interface uses Session:
val serverInterface = KiloInterface<Session>().apply {
    onConnected {
        // Do some initialization
        session.fooState = null
    }
    // Exceptions are passed through the network and re-created (re-thrown) on other side:
    on(cmdGetFoo) { session.fooState ?: throw IllegalStateException("foo is not yet set") }
    on(cmdSetFoo) { session.fooState = it }
}

// now create server using ktor (see ktor project for more):

val ns: NettyApplicationEngine = embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = {
    setupWebsocketServer(serverInterface) { Session() }
}).start(wait = false)

~~~

## Create a TCP / IP client and server

Using plain TCP/IP is even simpler, and it works way faster than websocket one, and is _the same
protected as `wss://` (and `ws://`) the variant above due to same kiloparsec encryption in both cases. Still, a TCP/IP
client is not available in JavaScript browser targets, and custom TCP ports could often be blocked by firewalls.

Documentation is available in samples here:

- [TCP/IP server creation](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-server/index.html)

- [TCP/IP client](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-client/index.html)

In short, there are two functions that implement asynchronous TCP/IP transport on all platforms buy JS:

- [acceptTcpDevice](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/accept-tcp-device.html?query=fun%20acceptTcpDevice(port:%20Int):%20Flow%3CInetTransportDevice%3E)
  to create a server

- [connectTcpDevice](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/connect-tcp-device.html)
  to connect to the server

## UDP client and server

Is very much straightforward, same as with TCP/IP:

- [UDP server creation](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/accept-udp-device.html)
- [Connect UDP client](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/connect-udp-device.html)

### UDP specifics

#### Command size

Each command invocation and result are packed in a separate UDP diagram using effective binary packing.
Thus, for the best results commands and results should be relatively short, best to fit into 240 bytes. While bigger
datagrams are often transmitted successfully, the probability of the effective transmission drops with the size.

Kiloparsec UDP transport does not retransmit not delivered packets. Use TCP/IP or websocket if it is a concern.

For the best results, we recommend
using [push](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-remote-interface/index.html#1558240250%2FFunctions%2F788909594)
for remote interfaces with UDP.

#### Timeouts

As Datagrams do not form protocol itself, kiloparsec issues pings when no data is circulated between parties.
When no pings are received long enough, the kiloparsec connection is closed. There are `maxInactivityTimeout` in all
relevant functions and constructors.

Client should not issue pings manually.

## Reusing code between servers

The same instance of
the [KiloInterface](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-interface/index.html?query=open%20class%20KiloInterface%3CS%3E%20:%20LocalInterface%3CKiloScope%3CS%3E%3E)
could easily be reused with all instances of servers with different protocols.

This is a common practice to create a business logic in a `KiloInterface`, then create a TCP/IP and Websocket servers
passing the same instance of the logic to both.

## Note on the server identification

We do not recommend relying on TLS (HTTPS://, WSS://) host identification solely; in the modern world there is
a high probability of attacks on unfriendly (in respect to at least some of your users) states to the SSL certificates
chain, in which case the [MITM attack] and spoofing will be undetected. Check
the [remoteId](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-client/remote-id.html?query=suspend%20fun%20remoteId():%20VerifyingPublicKey?)
in your client on each connection and provide the
safe [serverSecretKey](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-server/index.html)
when creating a server.

This will effectively protect against certificate chain spoofing in the case of the application installed from the
trusted source.

__Important note__. The web application could not be completely secured this way unless is loaded from the IP-address,
as the DNS could be spoofed the same, especially when used with `Cloudflare` or other CDN that can transparently
substitute the whole site. For applications, we strongly recommend not using CDN except your own, controlled ones. You
generally can't neither detect nor repel [MITM attack] performed from _any single cloudflare 'ray'_.

## See also:

- [Source documentation](https://code.sergeych.net/docs/kiloparsec/)
- [Project's WIKI](https://gitea.sergeych.net/SergeychWorks/kiloparsec/wiki)

# Details

It is not compatible with Parsec family and no more based on the Universa crypto library. To better fit
the modern state of threats and rate of cyber crimes, KiloParsec uses more encryption and random key exchange on each
and every connection (while parsec caches session keys to avoid time-consuming keys exchange). For the same reason,
keys cryptography for session is shifted to use ed25519 curves, which are supposed to provide agreeable strength with
enough speed to protect every connection with unique new keys. Also, we completely get rid of SHA2.

Kiloparsec also uses a denser binary format [bipack](https://gitea.sergeych.net/SergeychWorks/mp_bintools), no more
key-values,
which reveals much less on the inner data structure, providing advanced
typed RPC interfaces with kotlinx.serialization. There is also Rust
implementation [bipack_ru](https://gitea.sergeych.net/DiWAN/bipack_ru).
The architecture allows connecting the same functional interfaces to several various type channels at once.

Also, the difference from parsecs is that there are no more unencrypted layer commands available to users.
All RPC is performed over the encrypted connection.

# Technical description

Kiloparsec is a full-duplex fully async (coroutine-based) Remote Procedure Call protocol with typed parameters
and support for serializing exceptions (e.g. exception thrown while executing remote command will be caught and
rethrown at the caller context).

Kiloparsec is not REST, it _has advanced session mechanisms_ and built-in authentication based on the same curve keys.
Integrated tools to prevent MITM attacks include also non-transferred independently generated token that is calculated
independently on the ends and is never transferred with the network. Comparing it somehow (visually, with QR code, etc.)
could add a very robust guarantee of the connection safety and ingenuity.

Kiloparsec has a built-in completely asynchronous (coroutine-based top-down) transport layer based on TCP (JVM only as for
now) and the same async Websocket-based transport based on KTOR. Websocket client is multiplatform, though the server is
JVM only insofar.

# Licensing

This is a work in progress, not yet moved to the public domain;
you need to obtain a license from https://8-rays.dev or [Sergey Chernov]. For open source projects it will most be free
on some special terms.

It will be moved to open source; we also guarantee that it will be moved to open source immediately if the software
export restrictions are lifted. We do not support such practices here at 8-rays.dev.

[MITM]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack

[Sergey Chernov]: https://t.me/real_sergeych
[^1]: On some new Xiaomi phones we found problems with websocket binary frames, probably in ktor; use text frames
otherwise.