bipack_ru: Rust Bipack implementation

Bipack serialization format is a compact binary format that was created as a replacement for a BOSS format, more compact, faster, and not revealing inner data structure as much as possible. It is used in our next generation data protection systems, in DiWAN products as a base format and also in my new communication package like kiloparsec (and in underlying crypto2) which are intended to replace older Parsec protocols family to cope with modern crazy world challenges.

Important

due to growing tensions in the world, the main development is moved to our Gitea repository. We will try our best to make it available all around the world.

The most recent code will be there, and we encourage to use it also for issues/discussions.

Beta test

Use with serde:

Use bipack_ru:ser:to_bytes() and bipack_ru:de:from_bytes() functions:

fn test() -> Result<(),Error> {
    let src = HashSet::from(["foo", "bar", "buz"].map(|i| i.to_string()));
    let packed = to_bytes(&src)?;
    println!("{}", to_dump(&packed));

    let restored: HashSet<String> = from_bytes(&packed)?;
    println!("{:?}", restored);
    assert_eq!(src, restored);
}

serde module supports all Rust formats except floats/doubles which are not yet used and standardized by bipack format.

• All integers, signed and unsigned, are encoded with variable-length smartint.

Fixed-size encoding is available using custom de/serializers (exactly as in postcard format):

#[derive(Serialize)]
pub struct DefinitelyBigEndian {
    #[serde(with = "bipack_ru::fixint::be")]
    // or #[serde(with = "bipack_ru::fixint::le")]
    x: u16,
}

Fixed or variable length arrays?

When encoding arrays, the rule is simple:

• dynamic arrays (arrays, vectors, etc.) are encoded with size, as variable-length
• fixed-size arrays, like tuples (1,2,3,4,5), or something like [u32; 5] are encoded as fixed-size entities.

Direct usage (no serde)

The sample code (see src/lib.rs for more:)

fn test() {
    let mut data = Vec::<u8>::new();
    data.put_str("Hello, rupack!");
    println!("size ${}\n{}", data.len(), to_dump(&data));
    let mut src = SliceSource::from(&data);
    assert_eq!("Hello, rupack!", src.get_str().unwrap());
}

Tools and macros

• to_dump to convert binary slice into human-readable dump
• StringBuilder super minimalistic string builder (footprint).

The autodoc documentation is good enough already, so we do not repeat it here now.

How to

• just ad this package to your dependencies, it is on crates.io.

Big thanks

• to https://github.com/jamesmunns for the brilliant postcard format which was used as a source of inspiration for the Rust bindings.

License

For compliance with other modules this work is provided under APACHE 2.0 license a copy of which is included in the file LICENSE.