Dassie
Menu

Snapshot wire format

A core node that offers swaps signs a snapshot of what it swaps at: a bid and an ask for each settlement method its peers pay it on, each with a ladder of how far a swap can go before the price gets worse. This page describes every byte of it, for anyone who writes their own reader. Read a node's snapshot shows how to use one.

On this page
  1. Where snapshots come from
  2. Encoding
  3. Request
  4. Snapshot
  5. Checking a snapshot
  6. Pricing a swap
  7. Versions
  8. Test vectors

Where snapshots come from

A node makes a snapshot at most once a second while its quotes change, and otherwise every 30 seconds, and keeps the signed bytes. The same bytes reach readers two ways:

  • Its peers subscribe over the peer protocol, and the node pushes each new snapshot to them. That is how a node prices a swap with a peer, and how a market maker reads the nodes it trades through.
  • Anyone else sends a request over UDP, to port 7768 by default, at the host of the node's URL, and gets the snapshot back.

Two places name the UDP port, while the node answers there:

  • The discovery document, https://<host>/.well-known/dassie/node, as feedPort, beside the node's id, its publicKey and its slhDsaPublicKey, both in base64url.
  • The node's link state, in extension type 3, for tools that read link states.

Neither names a port the node doesn't serve. A node whose owner turned swaps off has no snapshot, and neither has a periphery node, one without a URL, which forwards nothing.

Encoding

Every datagram is canonical OER (ITU-T X.696) and starts with a version byte, today 2. Integers are big-endian and fixed in size. A list count is the count as a big-endian integer in as few bytes as it fits, after one byte that gives how many bytes it takes: zero entries is 01 00, two is 01 02. A CHOICE tag is one byte, 0x80 plus the tag number.

Every value has exactly one encoding, and a decoder refuses anything else: methods in another order, a method twice, a quote or a ladder not in its one form, or padding that isn't zero. So a snapshot's bytes are exactly what was signed. Every datagram fits in 1,200 bytes, which one packet carries on any path IPv6 runs on. 

Datagram ::= SEQUENCE {
  version  UInt8 (2),
  message  CHOICE {
    request  [0] zero octets, to exactly 1180 bytes in all,
    snapshot [1] SEQUENCE {
      sequence     UInt64,                        -- ms, only grows
      validUntil   UInt64,                        -- ms, after sequence
      grid         SEQUENCE (SIZE (8)) OF UInt16, -- drops in basis points,
                                                  -- ascending, each below 10000
      methods      SEQUENCE (SIZE (0..36)) OF SEQUENCE {
        tag        OCTET STRING (SIZE (4)),       -- ascending, each once
        bid        UInt32,                        -- a quote
        ask        UInt32,                        -- a quote
        bidLadder  OCTET STRING (SIZE (9)),       -- a ladder
        askLadder  OCTET STRING (SIZE (9))        -- a ladder
      },
      signature    OCTET STRING (SIZE (64))
    }
  }
}

The fixed sizes are written without a length. The layout is in crates/dassie-feed/src/snapshot.rs, and the numbers in number.rs.

Request

Always exactly 1,180 bytes, the size of the largest snapshot: 02, 80, then 1,178 zero bytes. The node answers with the snapshot it has, from the port you sent to. So it never sends an address more bytes than it got from it, and a request with a forged sender can't be used to flood someone else.

There is no subscription and no cookie: ask again whenever you want a newer snapshot. The node keeps nothing about who asked. A budget of 64 answers a second, after a burst of 64, bounds what all requests together cost it. A request gets no answer when:

  • it is of another size or version, or its padding isn't zero;
  • it comes from port 0, or from an address that is unspecified, multicast or broadcast;
  • the budget is spent for now;
  • the node has no snapshot that counts: it offers no swaps, is a periphery node, or its engine isn't running.

UDP drops packets now and then, so a reader that hears nothing waits a little and asks again.

Snapshot

02, 81, the fields below, then the 64-byte Ed25519 signature. A snapshot carries no key and no node ID: you know which node you asked.

sequence

When the snapshot was made, in milliseconds since the Unix epoch. While the node runs, each snapshot has a larger one, even if its clock went back. Nothing keeps it across a restart, so after a restart with the clock set back, you skip the node's snapshots until its clock passes the last one you took.

validUntil

When the snapshot stops counting, in milliseconds; always after sequence. Usually 90 seconds after it, three heartbeats, or earlier when a rate the node prices with ages out.

grid

Eight drops in price, in basis points, strictly ascending and each below 10,000 (100%). Each ladder has one amount for each drop. Nodes use 1, 3, 10, 30, 100, 300, 1,000 and 3,000: 0.01% to 30%. The grid travels in the snapshot, so a node can change it without a new version.

methods

The settlement methods the node's peers pay it on, at most 36, in ascending order of their tags, each once. A method the node offers but doesn't convert on right now, for want of a fresh rate, say, is listed with both sides closed, so you can tell it from one the node doesn't offer.

tag

The first 4 bytes of the SHA-256 of dassie-feed/method-tag followed by the scheme's ID. You can only use methods whose schemes you run, so you know their IDs and compute their tags:

Scheme Tag
hub-xrpl-testnet-xrp b0 fa 3f 74
hub-xrpl-testnet-dsusd 87 b5 e4 45

Schemes and hubs lists every scheme.

bid

The least the node gives, in its own accounting unit, for one unit of the method's ledger that comes in, its fee and spread included. Rounded down. A bid of zero means the node takes nothing in on this method now.

ask

The most the node asks, in its accounting unit, for one unit of the method's ledger that goes out. Rounded up. An ask of zero means the node pays nothing out on this method now. The accounting unit is the node's own, so only the ratio of a bid to an ask means anything.

bidLadder

For each drop of the grid, the most that can come in on the method with its last unit still given at least the bid less that drop, in units of the method's ledger.

askLadder

For each drop, the most that can go out on the method with its last unit still costing at most the ask over one less that drop. A closed side's ladder is all zero.

Quotes

A bid or an ask is a 32-bit float of Dassie's own. The top 8 bits are an exponent e and the low 24 a mantissa m, and the value is m × 2^(e − 150). The mantissa is at least 2^23, so each value has one form; the word 00 00 00 00 is zero, a closed side. For example, 7f 80 00 00 is 1, and 7f 80 20 c5 is 1.001 rounded up.

Ladders

A ladder is 9 bytes: 2 for its first amount, then a step for each of the 7 amounts after it.

  • The first amount is a 16-bit float: the top 6 bits an exponent e, the low 10 a mantissa m, and the value m × 2^e. The mantissa is at least 512 unless e is 0, and e is at most 54. So 03 e8 is 1,000, and 42 62 is 610 × 2^16, 39,976,960.
  • Each later amount is a step b up from the one before: floor(prev × 2^(b/16)), computed in integers as floor(prev × F[b mod 16] / 2^(62 − floor(b/16))), where F[k] = floor(2^(62 + k/16)). The 16 values of F are STEP_FRACTIONS in number.rs. A step of 10 (16) doubles the amount. Of the steps that give an amount, the snapshot uses the smallest.

So a ladder's amounts never go down. The node rounds every one of them down, so a ladder never says more than the node gives.

An example

A snapshot with two methods, from the test vectors: 160 bytes. 

# version, snapshot
02 81
# sequence 1790000000000, validUntil 1790000090000
00 00 01 a0 c4 50 6c 00
00 00 01 a0 c4 51 cb 90
# grid: 10, 25, 50, 100, 200, 400, 800, 1600 basis points
00 0a 00 19 00 32 00 64 00 c8 01 90 03 20 06 40
# two methods
01 02
# hub-xrpl-testnet-dsusd: bid 0.999, ask 1.001,
# ladders from 39,976,960, doubling at each drop
87 b5 e4 45  7e ff be 76  7f 80 20 c5
42 62 10 10 10 10 10 10 10  42 62 10 10 10 10 10 10 10
# hub-xrpl-testnet-xrp: bid 2.49, ask 2.51,
# ladders from 14,991,360, doubling at each drop
b0 fa 3f 74  80 9f 5c 28  80 a0 a3 d8
3b 93 10 10 10 10 10 10 10  3b 93 10 10 10 10 10 10 10
# signature
03 e5 3e 67 eb 6f 66 f9 b6 60 ce 79 68 8d 79 73
94 d3 44 aa f3 48 37 5b 5b 8e 8b 95 e1 85 2d 2f
57 5c c4 84 27 7e 44 d6 cc 0f 96 b5 2d 72 05 ad
a8 9a fc 60 19 7d 45 56 82 56 fd cb e3 e3 c7 0a

Checking a snapshot

The signature is Ed25519 by the node key, over the snapshot's object identifier, 1.3.6.1.4.1.54392.5.1399.3.2, followed by every byte before the signature, the version byte first. The identifier isn't sent. In OER it is these 14 bytes: 

0d 2b 06 01 04 01 83 a8 78 05 8a 77 03 02

It makes sure a snapshot's signature never passes for anything else the node signs. Before you act on a snapshot:

  1. Take datagrams only from the node's address and port.
  2. Decode it, and refuse it if it is of another version, not canonical, or breaks a rule above.
  3. Take it only if its sequence is after the last one you took and validUntil is still ahead, so a replayed or delayed snapshot never sets quotes back.
  4. Verify the signature by the ZIP-215 rules against the node's Ed25519 key, and refuse a key of small order. You learned both of the node's keys, and checked them against its ID, from its discovery document or its link state.

A snapshot is signed, not encrypted. It says nothing the node's packets wouldn't.

Why it works this way and why a snapshot carries no key.

Pricing a swap

A swap from method a to method b is priced at a's bid over b's ask. Then the ladders take a share off as the swap grows. Both are read by what has come in on a: the bid ladder by the amount itself, the ask ladder by what it is worth at the top price, amount × bid / ask, in b's units. While what came in is within the bid ladder's amount for drop di and the ask ladder's for drop dj, each unit in gets at least bid × (1 − di) × (1 − dj) / ask. A swap larger than either ladder's last amount isn't covered.

The node prices each packet at its last unit. So a swap sent as one packet per step of the ladders gets at least the sum of the steps, and a packet across several steps gets the rate of the step its last unit falls in, on all of it. In the example above, 50,000,000 units of DSUSD get at least 19,847,287 units of XRP sent one packet per step, and at least 19,801,018 sent as one packet. Every step rounds down, in integers, so every reader gets the same numbers. swap.rs has the walk.

Versions

Within a version, the layout never changes: readers run software of their own, outside Dassie's releases. A change is a new version byte, and its snapshot is signed under a new object identifier, so no signature passes for another version's. A datagram of any other version is refused.

Version 2, signed under …1399.3.2, is the snapshot. It took the place of the rate feed's version 2 before any release carried that version. The rate feed that 0.2.x nodes published was version 1, signed under …1399.3.1, and a 0.3.0 node refuses it. The bound on methods is part of the layout: the largest snapshot, with 36 methods, fits in one datagram, and one method more would not.

Why snapshots of quotes and ladders replaced a feed of rates, and why a node states its prices at all, in passive rebalancing.

Test vectors

testdata/feed/snapshots.json holds method tags, quotes, first amounts, the step table, ladders, the request, snapshots from a heartbeat with no methods to the largest the bounds allow (1,180 bytes), swaps priced from one, and datagrams that must be refused, each with its reason. All are signed with a fixed key. A changed vector means a new version.

A kopje of granite blocks against the sun, in veld grass; one dassie climbs its slab, one keeps lookout on top inside the sun, and one sleeps on the next rock; on a wide screen, a pair sits on a kopje to the left