# Top-of-Book over RF Besides the request/response protocol served over WebSocket, the **Market Data Adapter** can push top-of-book updates as bare **UDP datagrams**. Each datagram carries exactly one top-of-book event in a compact binary encoding, with no framing, no handshake, and no acknowledgement. The encoding is compact enough to travel over **RF (radio) links**, which is what it was built for. An RF hop between two regions can beat the fibre route, and top-of-book is the one topic worth the scarce bandwidth on such a link. --- ## Two ways to receive it ### Through a regional proxy This is the normal setup, and the one to assume unless you have agreed otherwise. CryptoStruct operates proxy services in the target region. A proxy consumes the RF/UDP stream and forwards the updates to you over a regular WebSocket connection. Nothing changes on your side: - Same [protocol](protocol.md), same login, same subscriptions, same choice of JSON or [SBE](sbe.md). - Same delivery guarantees, because the transport to you stays TCP-based. - Every market-data topic stays available on that one connection, not just top-of-book. - Top-of-book simply arrives earlier. You profit from the RF path automatically. Nothing on this page applies to you in that case. It is worth reading only if you are considering the direct path below. ### Directly over UDP Reading the datagrams yourself removes the proxy hop, which is as low as the latency goes. In exchange you take on everything the proxy would otherwise absorb: - Packet loss, duplication, and reordering, which you have to detect yourself (see [Detecting packet loss](#detecting-packet-loss)). - Top-of-book only, for a fixed set of instruments, in selected regions. - The decoder described in the rest of this page. Worth it when you have measured that the proxy hop costs you something that matters. Otherwise the proxy is the better trade. > [!WARNING] > UDP gives no delivery and no ordering guarantees. Packets can be lost, > duplicated, or reordered, and nothing is resent. Always track the > sequence number described in [Detecting packet > loss](#detecting-packet-loss). --- ## Availability The RF infrastructure exists in **selected regions only**, and link capacity is scarce enough that each one carries a **fixed set of instruments**. Contact CryptoStruct to find out what is available at your location. For the direct path there is no login and no subscription. You agree a **host, a port, and a set of instruments** with CryptoStruct, and the Market Data Adapter pushes those instruments to that address for as long as it runs. What this means in practice: - **Only the agreed instruments arrive.** The set is fixed up front. You cannot request an instrument at runtime, and nothing outside the set is ever broadcast. To add or remove instruments, contact CryptoStruct. - **The feed is not tied to a connection.** Packets start arriving as soon as the market moves, whether or not you have anything else connected, and there is nothing to reconnect after an outage on your side. - **Receiving is a plain UDP read.** Bind a UDP socket to the agreed port and read datagrams. Nothing else is required, and nothing is sent back. For anything outside the agreed set, or for any other market-data topic, use a [WebSocket connection](accessing-data.md) with the regular [subscription](protocol.md#subscription) flow. Running both side by side is normal: UDP for the instruments you care about most, the regular connection for everything else. --- ## Packet layout One event per datagram. A UDP datagram is self-delimiting. All multi-byte integers are **big-endian** (network byte order). > [!WARNING] > This is the opposite of [SBE](sbe.md), which is little-endian. Only the > decimal mantissa is big-endian in both encodings. ### Header A fixed **30-byte** header: {.compact} | Offset | Size | Field | Type | Description | |--------|------|------------------------|----------|------------------------------------------------| | `0` | `1` | `protocolVersion` | `uint8` | Always `1` | | `1` | `1` | `messageType` | `uint8` | Always `6` (top-of-book, as in the JSON protocol) | | `2` | `4` | `instrumentId` | `uint32` | Instrument ID | | `6` | `8` | `sendTimeAndSequence` | `uint64` | Send time and packet sequence number, packed (see below) | | `14` | `8` | `eventId` | `uint64` | ID of this market-data event | | `22` | `4` | `exchangeTimestampAge` | `uint32` | Send time minus exchange timestamp, in µs; `0xFFFFFFFF` if not available | | `26` | `4` | `adapterTimestampAge` | `uint32` | Send time minus adapter timestamp, in µs | ### Send time and sequence number `sendTimeAndSequence` packs two values into one 64-bit integer: {.compact} | Bits | Field | Description | |-----------|------------------|----------------------------------------------------| | `63`–`12` | `sendTime` | Time the packet was sent, in µs since epoch (52 bits) | | `11`–`0` | `sequence` | Packet sequence number, `0`–`4095` (12 bits) | ``` sendTime = sendTimeAndSequence >> 12 sequence = sendTimeAndSequence & 0xFFF ``` 52 bits of microseconds last until the year 2112. Because the send time occupies the upper bits, the packed value as a whole increases monotonically from packet to packet. See [Detecting packet loss](#detecting-packet-loss) for how to use both parts. ### Decimal fields The header is followed by exactly four variable-length decimal fields, in this fixed order: 1. bid price 2. bid quantity 3. ask price 4. ask quantity Each field is encoded as: {.compact} | Size | Field | Type | Description | |------|------------|---------|---------------------------------------------------| | `1` | `scale` | `uint8` | Decimal scale, `0`–`31` | | `1` | `length` | `uint8` | Length `N` of the mantissa in bytes; `0` = field not available | | `N` | `mantissa` | bytes | Unscaled value, big-endian two's complement | The value is reconstructed exactly as for SBE decimals: ``` number = mantissa * (10 ^ -scale) ``` See [Decimal encoding](sbe.md#decimal-encoding) for the full explanation. The mantissa is written at its minimal width, so `N` is at most `16`. A present value of zero is encoded with `length = 1` and a single `0x00` mantissa byte, which is what distinguishes it from an absent field. ### Absent sides A side (bid or ask) counts as present only if **both** its price and its quantity carry a mantissa. An absent side is encoded as four zero bytes: `scale = 0` and `length = 0` for its price and for its quantity. Unlike the JSON and SBE top-of-book events, where the order of the two sides is undefined, the UDP layout is positional: bid always comes before ask. ### Packet size {.compact} | Case | Size | |-------------------|-------------------------------------| | Both sides absent | `38` bytes (30 header + 2 × 4) | | Typical | `43`–`58` bytes | | Maximum | `102` bytes (30 header + 4 × 18) | --- ## Example Instrument `42`, send time `1700000000000000` µs, sequence number `999`, event ID `100`, exchange timestamp `1500` µs and adapter timestamp `40` µs before the send time, bid `99.50` for `1.25`, ask `99.51` for `4.00`. 45 bytes in total: {.compact} | Offset | Bytes | Meaning | |--------|---------------------------|--------------------------------------------| | `0` | `01` | Protocol version `1` | | `1` | `06` | Message type `6`, top-of-book | | `2` | `00 00 00 2A` | Instrument ID `42` | | `6` | `60 A2 41 81 E4 00 03 E7` | Send time `1700000000000000` µs, sequence `999` | | `14` | `00 00 00 00 00 00 00 64` | Event ID `100` | | `22` | `00 00 05 DC` | Exchange timestamp `1500` µs before send time → `1699999999998500` µs | | `26` | `00 00 00 28` | Adapter timestamp `40` µs before send time → `1699999999999960` µs | | `30` | `02 02 26 DE` | Bid price: scale `2`, mantissa `9950` → `99.50` | | `34` | `02 01 7D` | Bid quantity: scale `2`, mantissa `125` → `1.25` | | `37` | `02 02 26 DF` | Ask price: scale `2`, mantissa `9951` → `99.51` | | `41` | `02 02 01 90` | Ask quantity: scale `2`, mantissa `400` → `4.00` | --- ## Detecting packet loss The 12-bit `sequence` is assigned by the broadcaster and increments by exactly `1` per packet sent, wrapping from `4095` back to `0`. There is **one sequence per feed**, shared by all instruments sent to your address, not one per instrument. It starts at `0` when the adapter starts. How to use it: - Track the last seen `sequence` for the feed. - Compute the distance modulo 4096: `gap = (sequence - last) & 0xFFF`. - `gap == 1` means the packet is the expected next one. - Any other value means packets were lost, or this packet is a duplicate or arrived out of order. To tell loss apart from duplicates and reordering, compare the full 64-bit `sendTimeAndSequence` value with the last one seen. It increases monotonically, so a value that is not larger than the last one belongs to a duplicate or a reordered packet, and should be dropped. The send time also covers what the 12-bit sequence cannot: a gap of 4096 packets or more, and adapter restarts, which reset the sequence to `0` but never move the send time backwards. The sequence number is **not** the market-data event ID. `eventId` identifies the market-data event and is carried separately in the same packet, so a gap in `eventId` does not imply packet loss (not every event is broadcast) and a gap in `sequence` does. --- ## Timestamps Timestamps on the wire have **microsecond** resolution, not the nanoseconds of the JSON and SBE protocols. The exchange and adapter timestamps have the same meaning as in the [JSON protocol](protocol.md#common), but are not sent as absolute values. Each is sent as its age relative to `sendTime`, which fits into 32 bits: ``` exchangeTimestamp = sendTime - exchangeTimestampAge adapterTimestamp = sendTime - adapterTimestampAge ``` Details: - An `exchangeTimestampAge` of `0xFFFFFFFF` means the exchange did not supply a timestamp. - Ages are capped at `0xFFFFFFFE` µs (about 71 minutes). An older timestamp decodes as exactly that far before the send time. - A timestamp ahead of the send time, for example because of exchange clock skew, is sent as age `0` and decodes as equal to the send time. `sendTime` is taken when the packet leaves the broadcaster. Comparing it with your own receive time gives the one-way latency of the link, provided the clocks are synchronised. --- ## Validation Reject a datagram when: - It is shorter than the 30-byte header. - `protocolVersion` is not `1`. - `messageType` is not `6`. - A `length` byte exceeds `16`. - A `length` byte claims more mantissa bytes than the datagram still holds, or the variable section is cut mid-field. Because packets arrive unsolicited from the network, treat every one of these as a malformed packet to drop and count, not as a fatal error. --- ## Comparison with the proxy path Both paths carry the same RF-accelerated top-of-book data. They differ in what you have to handle. {.compact} | Aspect | Regional proxy (JSON / SBE) | Direct UDP | |--------------|-----------------------------------|----------------------------------| | Transport | WebSocket | UDP datagram | | Delivery | Guaranteed, ordered | Best effort, unordered | | Setup | Login, then subscribe | None, agreed in advance | | Topics | All market-data topics | Top-of-book only | | Instruments | Any, chosen at runtime | Fixed set, agreed in advance | | Framing | WebSocket frames | None, one event per datagram | | Byte order | Little-endian (SBE) | Big-endian | | Event IDs | `eventId` and `prevEventId` | `eventId` and packet sequence, no `prevEventId` | | Timestamps | Nanoseconds, absolute | Microseconds, relative to send time | | Latency | RF path plus one proxy hop | RF path only | Because no previous event ID is transmitted, a UDP receiver cannot chain events the way a book-building client does. This is not a limitation in practice: each packet carries the complete top of book, so the latest packet is always sufficient on its own. --- ## See Also - [Accessing Data](accessing-data.md). How to connect and subscribe. - [Protocol](protocol.md). Market data protocol specification. - [SBE Encoding](sbe.md). Simple Binary Encoding and decimal encoding.