Message  Pack
1 Introduction
2 An example session
3 Message  Pack API
3.1 Data types
3.1.1 Packable types
Packable
packable?
3.1.2 Message  Pack extension type
Ext
ext
3.2 Packing
pack
3.3 Unpacking
unpack
6.12

MessagePack

Alejandro Sanchez <[email protected]>

 (require msgpack) package: msgpack

MessagePack is a binary serialisation format focused on speed and size. This library allows you to serialise (pack) and de-serialise (unpack) Racket objects.

Source code: https://gitlab.com/HiPhish/MsgPack.rkt

    1 Introduction

    2 An example session

    3 MessagePack API

      3.1 Data types

        3.1.1 Packable types

        3.1.2 MessagePack extension type

      3.2 Packing

      3.3 Unpacking

1 Introduction

When two processes want to exchgange data they need to agree on a protocol for serialising and de-serialising said data. MessagePack is a protocol designed with speed and size in mind; this means that serialised data should be as small as possible and the recipient should be able to de-serialise the data as quickly as possible. The flipside of this is that the protocol is not easily readable to humans, but this is not a concern if the data is only meant to be exchanged beetween processes anyway.

The API of this implementation follows the naming conventions of MessagePack: we call the serialising process packing and the de-serialising process unpacking. Integers have different length and can be either signed (intN) or unsigned (uintN), where N is the length of the integer in bytes.

MessagePack objects are converted to Racket objects and vice-versa according to the following rules:

MessagePack type

Racket type

nil

(void)

true

#t

false

#f

positive-fixint, negative-fixint, uint8, uint16, uint32, uint64, int8, int16, int32, int64

integer

fixstr, str8, str16, str32

string

bin8, bin16, bin32

bytes

float16, float32

real

fixarray, array16, array32

vector list

fixmap, map16, map32

hash

fixext1, fixext2, fixext4, fixext8, fixext16, ext8, ext16, ext32

ext

When there is more than one Racket type listed unpack will return a value of the first type listed.

2 An example session

Here we have an object in Racket which we wish to pack. The object is a vector of various other packable Racket objects. Objects are packed to ports, usually these ports point to files or network connections, but here we will use byte strings as ports for the sake of simplicity.

; Import the library first
> (require msgpack)
; Here is some data we want to pack: a vector of numbers,
; nothing, another vector and a string.
> (define hodgepodge (vector 1 2 (void) '#(3 #t) "foo"))
; Use a byte string as the output port
> (define packed (call-with-output-bytes (λ (out) (pack hodgepodge out))))
; The entire hodgepodge has now been packed to binary data
> packed

#"\225\1\2\300\222\3\303\243foo"

; If we want our original hodgepodge back we need to unpack it
> (define unpacked (call-with-input-bytes packed (λ (in) (unpack in))))
> unpacked

'#(1 2 #<void> #(3 #t) "foo")

Packing and unpacking are the primitive operations associated with the MessagePack format, more complex tasks like sending and receiving RPC messages can then be implemented on top of this library.

3 MessagePack API

3.1 Data types

3.1.1 Packable types

 (require msgpack/packable) package: msgpack

syntax

Packable

Union of all packable types for use with Typed Racket. Use this as the most general type for objects you can send to pack or receive from unpack.

procedure

(packable? x)  boolean?

  x : Any
True if x can be packed.

3.1.2 MessagePack extension type

 (require msgpack/ext) package: msgpack

syntax

Ext

The type of an ext structure for use with Typed Racket.

struct

(struct ext (type data))

  type : integer?
  data : bytes?
Represents a MessagePack extension type, a pair of a signed 8-bit type integer and a data byte string. The type name for Typed Racket is Ext.

3.2 Packing

 (require msgpack/pack) package: msgpack

procedure

(pack datum out)  any

  datum : any/c
  out : (and/c output-port? (not/c port-closed?))
Pack datum into the out port. The type to pack datum to will be determined automatically to use the least amount of space.

3.3 Unpacking

 (require msgpack/unpack) package: msgpack

procedure

(unpack in)  any

  in : (and/c input-port? (not/c port-closed?))
Unpack a datum from in. At least one byte is consumed in the process to read the tag, more bytes are consumed as needed by the type of data.