277 lines
12 KiB
Markdown
277 lines
12 KiB
Markdown
# Kiloparsec
|
|
|
|
__Recommended version is `0.4.1`: to keep the code compatible with current and further versions we
|
|
ask to upgrade to `0.4.2` at least.__ Starting from this version some package names are changed for
|
|
better clarity and fast UDP endpoints are added.
|
|
|
|
The new generation of __PARanoid SECurity__ protocol, advanced, faster, more secure. It also allows connecting any "
|
|
block device" transport to the same local interface. Out if the box it
|
|
provides the following transports:
|
|
|
|
| name | JVM | JS | native |
|
|
|-------------------|--------|----|--------|
|
|
| TCP/IP server | ✓ | | 0.2.6+ |
|
|
| TCP/IP client | ✓ | | 0.2.6+ |
|
|
| UDP server | 0.3.2+ | | 0.3.2+ |
|
|
| UDP client | 0.3.2+ | | 0.3.2+ |
|
|
| Websockets server | ✓ | | |
|
|
| Websockets client | ✓ | ✓ | ✓ |
|
|
|
|
### Note on version compatibility
|
|
|
|
Version 0.5.1 could be backward incompatible due to upgrade of the crypto2.
|
|
|
|
Protocols >= 0.3.0 are not binary compatible with previous version due to more compact binary
|
|
format. The format from 0.3.0 onwards is supposed to keep compatible.
|
|
|
|
#### ID calculation algorithm is changed since 0.4.1
|
|
|
|
We recommend to upgrade to 0.4+ ASAP as public/shared key id derivation method was changed for even higher security.
|
|
|
|
### Supported native targets
|
|
|
|
- iosArm64, iosX64
|
|
- macosArm64, macosArm64
|
|
- linuxArm64, linuxX64
|
|
|
|
### Non-native targets
|
|
|
|
- JS (browser and Node.js)
|
|
- JVM (android, macOS, windows, linux, everywhere where JRE is installed)
|
|
|
|
## TCP/IP and UDP transports
|
|
|
|
These are the fastest based on async socket implementation of ktor client. They works everywhere but JS target as
|
|
there is currently no widely adopted sockets for browser javascript.
|
|
|
|
While UDP is faster than TCP/IP, it is less reliable, especially with commands and return values that serializes to more than 240 bytes approx, and has no retransmission facilities (use TCP!). UDP though shines when all you need is to [push](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-remote-interface/push.html) with little or no data in it.
|
|
|
|
## Websockets server
|
|
|
|
While it is much slower than TCP or UDP, it is still faster than any http-based API; it uses binary frames based on
|
|
the Ktor server framework to easily integrate with web services. We recommend using it instead of a classic HTTP API as
|
|
it beats it in terms of speed and server load even with HTTP/2.
|
|
|
|
We recommend to create the `KiloInterface<S>` instance and connect it to the websockets and tcp servers in real
|
|
applications to get easy access from anywhere.
|
|
|
|
## Websocket client
|
|
|
|
It is slower than TCP or UDP, but it works on literally all platforms. See the sample below.
|
|
|
|
# Usage
|
|
|
|
The library should be used as maven dependency, not as source.
|
|
|
|
## Adding dependency
|
|
|
|
### Declare maven repository:
|
|
|
|
Add the private repository to your `build.gradle.kts`, like:
|
|
|
|
```kotlin
|
|
repositories {
|
|
maven("https://gitea.sergeych.net/api/packages/SergeychWorks/maven")
|
|
}
|
|
```
|
|
|
|
### Add dependency block
|
|
|
|
It could be, depending on your project structure, something like:
|
|
|
|
```kotlin
|
|
val commonMain by getting {
|
|
dependencies {
|
|
api("net.sergeych:kiloparsec:0.4.1")
|
|
}
|
|
}
|
|
```
|
|
|
|
## Create shared interface for your server and the client
|
|
|
|
It could be a multiplatform library that exports it or just a shared or copied source file declaring structures
|
|
and functions available, like:
|
|
|
|
```kotlin
|
|
// Api.kt
|
|
|
|
@Serializable
|
|
class FooArgs(val text: String, val number: Int = 42)
|
|
|
|
// Server-side interface
|
|
val cmdSetFoo by command<FooArgs, Unit>()
|
|
val cmdGetFoo by command<Unit, FooArgs>()
|
|
val cmdPing by command<String, String>()
|
|
val cmdCheckConnected by command<Unit, Boolean>()
|
|
|
|
// client-side interface (called from the server)
|
|
val cmdPushClient by command<String, Unit>()
|
|
```
|
|
|
|
## Call it from the client:
|
|
|
|
Remember, we need to implement client interface `cmdPushClient` in our example, so we need to provide
|
|
local interface too:
|
|
|
|
```kotlin
|
|
// Unit: no session on the client:
|
|
val client = websocketClient<Unit>("wss://your.host.com/kp") {
|
|
// This is server-callable function we export:
|
|
on(cmdPushClient) {
|
|
"server push: $it"
|
|
}
|
|
}
|
|
|
|
// If we want to collect connected state changes (this is optional)
|
|
launch {
|
|
client.connectedStateFlow.collect {
|
|
if (it)
|
|
println("I am connected")
|
|
else
|
|
println("trying to connect...")
|
|
}
|
|
}
|
|
|
|
// now we can call server's functions
|
|
client.call(cmdSetFoo, FooArgs("bar", 117))
|
|
assertEquals(FooArgs("bar", 117), client.call(cmdGetFoo))
|
|
|
|
```
|
|
|
|
## Create ktor-based server
|
|
|
|
Normally server side needs some session. It is convenient and avoid sending repeating data on each request speeding up
|
|
the protocol. With KILOPARSEC, it is rather basic operation:
|
|
|
|
~~~kotlin
|
|
// Our session just keeps Foo for cmd{Get|Set}Foo:
|
|
data class Session(var fooState: FooArgs? = null)
|
|
|
|
// Let's now provide interface we export, it will be used on each connection automatically:
|
|
|
|
// Note server interface uses Session:
|
|
val serverInterface = KiloInterface<Session>().apply {
|
|
onConnected {
|
|
// Do some initialization
|
|
session.fooState = null
|
|
}
|
|
// Exceptions are passed through the network and re-created (re-thrown) on other side:
|
|
on(cmdGetFoo) { session.fooState ?: throw IllegalStateException("foo is not yet set") }
|
|
on(cmdSetFoo) { session.fooState = it }
|
|
}
|
|
|
|
// now create server using ktor (see ktor project for more):
|
|
|
|
val ns: NettyApplicationEngine = embeddedServer(Netty, port = 8080, host = "0.0.0.0", module = {
|
|
setupWebsocketServer(serverInterface) { Session() }
|
|
}).start(wait = false)
|
|
|
|
~~~
|
|
|
|
## Create TCP/IP client and server
|
|
|
|
Using plain TCP/IP is even simpler, and it works way faster than websocket one, and is _the same
|
|
protected as `wss://` (and `ws://`) variant above due to same kiloparsec encryption in both cases. Still, a TCP/IP
|
|
client is not available in Javascript browser targets and custom TCP ports could often be blocked by firewalls.
|
|
|
|
Documentation is available in samples here:
|
|
|
|
- [TCP/IP server creation](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-server/index.html)
|
|
|
|
- [TCP/IP client](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-client/index.html)
|
|
|
|
In short, there are two functions that implements asynchronous TCP/IP transport on all platforms buy JS:
|
|
|
|
- [acceptTcpDevice](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/accept-tcp-device.html?query=fun%20acceptTcpDevice(port:%20Int):%20Flow%3CInetTransportDevice%3E) to create a server
|
|
|
|
- [connectTcpDevice](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/connect-tcp-device.html) to connect to the server
|
|
|
|
## UDP client and server
|
|
|
|
Is very much straightforward, same as with TCP/IP:
|
|
|
|
- [UDP server creation](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/accept-udp-device.html)
|
|
- [Connect UDP client](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec.adapter/connect-udp-device.html)
|
|
|
|
### UDP specifics
|
|
|
|
#### Command size
|
|
|
|
Each command invocation and result are packed in a separate UDP diagram using effective binary packing.
|
|
Thus, for the best results commands and results should be relatively short, best to fit into 240 bytes. While bigger datagrams are often transmitted successfully, the probability of the effective transmission drops with the size.
|
|
|
|
Kiloparsec UDP transport does not retransmit not delivered packets. Use TCP/IP or websocket if it is a concern.
|
|
|
|
For the best results we recommend using [push](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-remote-interface/index.html#1558240250%2FFunctions%2F788909594) for remote interfaces with UDP.
|
|
|
|
#### Timeouts
|
|
|
|
As Datagrams do not form protocol itself, kiloparsec issues pings when no data is circulated between parties.
|
|
When no pings are received long enough, kiloparsec connection is closed. There are `maxInactivityTimeout` in all
|
|
relevant functions and constructors.
|
|
|
|
Client should not issue pings manually.
|
|
|
|
## Reusing code between servers
|
|
|
|
The same instance of the [KiloInterface](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-interface/index.html?query=open%20class%20KiloInterface%3CS%3E%20:%20LocalInterface%3CKiloScope%3CS%3E%3E) could easily be reused with all instances of servers with different protocols.
|
|
|
|
This is a common proactive to create a business logic in a `KiloInterface`, then create a TCP/IP and Websocket servers passing the same instance of the logic to both.
|
|
|
|
## Note on the server identification
|
|
|
|
We do not recommend to rely on TLS (HTTPS://, WSS://) host identification solely, in the modern world there is
|
|
a high probability of attacks on unfriendly (in respect to at least some of your users) states to the SSL certificates
|
|
chain, in which case the [MITM attack] and spoofing will be undetected. Check the [remoteId](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-client/remote-id.html?query=suspend%20fun%20remoteId():%20VerifyingPublicKey?) in your client on each connection and provide the safe [serverSecretKey](https://code.sergeych.net/docs/kiloparsec/kiloparsec/net.sergeych.kiloparsec/-kilo-server/index.html) when creating a server.
|
|
|
|
This will effectively protect against certificate chain spoofing in the case of the application installed from the trusted source.
|
|
|
|
__Important note__. The web application could not be completely secured this way unless is loaded from the IP-address, as the DNS could be spoofed the same, especially when used with `Cloudflare` or other CDN that can transparently substitute the whole site. For applications, we strongly recommend not to use CDN except your own, controlled ones. You generally can't neither detect nor repel [MITM attack] performed from _any single cloudflare 'ray'_.
|
|
|
|
## See also:
|
|
|
|
- [Source documentation](https://code.sergeych.net/docs/kiloparsec/)
|
|
- [Project's WIKI](https://gitea.sergeych.net/SergeychWorks/kiloparsec/wiki)
|
|
|
|
# Details
|
|
|
|
It is not compatible with parsec family and no more based on an Universa crypto library. To better fit
|
|
the modern state of threats and rate of cyber crimes, KiloParsec uses more encryption and random key exchange on each
|
|
and every connection (while parsec caches session keys to avoid time-consuming keys exchange). For the same reason,
|
|
keys cryptography for session is shifted to use ed25519 curves which are supposed to provide agreeable strength with
|
|
enough speed to protect every connection with a unique new keys. Also, we completely get rid of SHA2.
|
|
|
|
Kiloparsec also uses a denser binary format [bipack](https://gitea.sergeych.net/SergeychWorks/mp_bintools), no more
|
|
key-values,
|
|
which reveals much less on the inner data structure, providing advanced
|
|
typed RPC interfaces with kotlinx.serialization. There is also Rust
|
|
implementation [bipack_ru](https://gitea.sergeych.net/DiWAN/bipack_ru).
|
|
The architecture allows connecting same functional interfaces to several various type channels at once.
|
|
|
|
Also, the difference from parsecs is that there are no more unencrypted layer commands available to users.
|
|
All RPC is performed over the encrypted connection.
|
|
|
|
# Technical description
|
|
|
|
Kiloparsec is a full-duplex fully async (coroutine based) Remote Procedure Call protocol with typed parameters
|
|
and support for serializing exceptions (e.g. exception thrown while executing remote command will be caught and
|
|
rethrown at the caller context).
|
|
|
|
Kiloparsec is not REST, it _has advanced session mechanisms_ and built-in authentication based on the same curve keys.
|
|
Integrated tools to prevent MITM attacks include also non-transferred independently generated token that is calculated
|
|
independently on the ends and is never transferred with the network. Comparing it somehow (visually, with QR code, etc.)
|
|
could add a very robust guarantee of the connection safety and ingenuity.
|
|
|
|
Kiloparsec has built-in completely asynchronous (coroutine based top-down) transport layer based on TCP (JVM only as for
|
|
now) and the same async Websocket-based transport based on KTOR. Websocket client is multiplatform, though the server is
|
|
JVM only insofar.
|
|
|
|
# Licensing
|
|
|
|
This is work in progress, not yet moved to public domain;
|
|
you need to obtain a license from https://8-rays.dev or [Sergey Chernov]. For open source projects it will most be free on some special terms.
|
|
|
|
It will be moved to open source; we also guarantee that it will be moved to open source immediately if the software export restrictions will be lifted. We do not support such practices here at 8-rays.dev.
|
|
|
|
[MITM]: https://en.wikipedia.org/wiki/Man-in-the-middle_attack
|
|
[Sergey Chernov]: https://t.me/real_sergeych |