Troubleshooting

Start with dassie status: it gives a hint wherever something needs you, and Check on your node says what each one means. This page goes further, for the problems people run into most.

Reading the journal

The node logs to systemd's journal. These show its recent lines, and follow new ones as they come:

$journalctl -u dassie -n 50
$journalctl -u dassie -f

Some lines are worth knowing:

• listening, at every start, names the version, the HTTP and HTTPS ports, the QUIC endpoint (quic=) and the UDP socket that answers requests for the node's snapshot (feed=, or none). It is the quickest way to see what the node listens on: journalctl -u dassie | grep listening.
• could not start comes with the reason the node stopped. See The node does not start.
• no bootstrap nodes: the node runs alone means DASSIE_BOOTSTRAP_NODES is set to []. The testnet's bootstrap nodes are built in, so remove the variable to join the testnet (Configure your node).
• It is for development; unset it on a public node ends a warning about a dev switch: see Dev switches.

To hear more from one part of the node, raise its log level with DASSIE_LOG, such as dassie::peer=debug for everything about peers: How much it logs.

dassie cannot reach the node

The dassie command talks to the node over a socket, /run/dassie.sock. It says which of two things went wrong:

• You may not use the Dassie daemon's socket: you are not in the group dassie-users yet. The installer adds the user who ran it, but the change counts only from your next login. Log out and in again, or use sudo.
• The Dassie daemon is not running: start it with sudo systemctl start dassie. If it does not stay up, read on.

The node does not start

The node checks everything before it starts, and if something is wrong it stops with a message rather than run half started. Find it with:

$systemctl status dassie
$journalctl -u dassie -n 50

The most common messages:

• A variable's name, then what is wrong with it, such as DASSIE_UPLINKS has an invalid node ID: fix or remove it in your drop-in, then restart. Configure your node shows where.
• A socket unit failed, as systemctl status dassie shows: another program holds one of the node's ports. Nodes need TCP 80 and 443 and UDP 443 for themselves, and dassie.service doesn't start while UDP 443 is taken, since QUIC is the only way nodes talk. A web server on the same machine is the usual cause, and some web servers take UDP 443 for HTTP/3 by default. Run sudo ss -tulpn to see who holds which port. UDP 7768 is different: without it the node starts all the same.
• another Dassie node runs on this data directory: a second node was started on the same data, perhaps by hand with dassie daemon and DASSIE_STATE_DIRECTORY set to the node's directory. Stop it; one node runs per data directory.
• the database is from a newer version of Dassie: you went back to an older version after a newer one changed the database. Put back the copy the newer version kept, or update again: Updates and rollback.
• a rollback stopped half way: the message says which files to put back to return to the database from before the update. Updates and rollback explains it.
• this database was made by a release of Dassie for the early testnet: the node's data is from a 0.2 release, before the testnet was reset, and this version cannot read it, nor the phrase that made it. Move the data aside, then configure and set up the node again, with a new phrase:

  $sudo systemctl stop dassie
  $sudo mv /var/lib/private/dassie /var/lib/private/dassie-0.2
  $sudo systemctl start dassie
  $sudo dassie init

  A node from the 0.2 releases says more.

Certificates

sudo dassie init gets a certificate from Let's Encrypt. Let's Encrypt checks that you control the domain by fetching a file from your node over plain HTTP, on port 80. When that fails, dassie init says The certificate could not be obtained, with Let's Encrypt's reason; a renewal that fails later shows on the Certificate line of dassie status.

Check, in this order:

1. The name points at this server. Look the name up and compare it with the server's public address: dig +short node.example.com. Remember both A (IPv4) and AAAA (IPv6): Let's Encrypt tries IPv6 when there is an AAAA record, so a stale one fails the check.

2. Port 80 is open in the server's firewall and in your cloud provider's. From another machine, curl -I http://node.example.com/ should get an answer from the node, a redirect to HTTPS.

3. Nothing else holds port 80. The node takes port 80 itself, through dassie-http.socket; a web server on the same machine keeps it from starting.

Then run sudo dassie init again. Let's Encrypt limits how often a name may fail. While you try things out, sudo dassie init --acme-staging uses its staging service, whose certificates browsers don't trust but whose limits are generous. The node keeps the server it got its certificate from, and renews there, so once staging works, name Let's Encrypt's production server, in a terminal:

$sudo dassie init --acme-directory https://acme-v02.api.letsencrypt.org/directory

It finds the staging certificate and asks whether to skip getting one: answer No. Without the flag it would order from staging again. Run from a script, it orders nothing while the node has a certificate. More in Certificates.

UDP and QUIC

Nodes talk to each other over QUIC only, which runs on UDP port 443. Many firewalls, and many cloud providers' presets for "HTTPS", open only TCP 443. A node whose UDP 443 is closed runs, serves its wallet and gets its certificate, but no other node can reach it: it is not listed, and it gets no peers.

1. Check that the node listens on UDP 443: the listening line in the journal names it, as quic=[::]:443.

   $journalctl -u dassie | grep listening

2. Open UDP 443 in the server's firewall, for example:

   # ufw (Debian, Ubuntu)
   $sudo ufw allow 443/udp
   # firewalld (Fedora, CentOS Stream)
   $sudo firewall-cmd --permanent --add-port=443/udp
   $sudo firewall-cmd --reload

3. Open it in your cloud provider's firewall or security group too: an inbound rule for UDP, port 443, from anywhere.

4. Restart the node, so that it registers again at once: sudo systemctl restart dassie.

If you cannot open UDP on this server, run it as a periphery node instead, without a domain name: it reaches the network through connections it opens itself, and where UDP is blocked, through QUIC over a WebSocket on its uplinks' HTTPS port, through an HTTP proxy where the network sets one. Only a node without a domain name falls back this way. See A server without a public IP.

Why it works this way: QUIC is the only transport between nodes.

Not listed

dassie status says not a member yet, listed by 0 of 5 bootstrap nodes, counting the testnet's five, or those you named in DASSIE_BOOTSTRAP_NODES. A node becomes a member of the network once more than half of the bootstrap nodes list it. It registers with each of them when it starts, and each one first checks that it can reach your node at its domain name: over QUIC, on UDP 443, and over HTTPS, on TCP 443, where it fetches your node's discovery document and needs a certificate a public authority issued.

• Right after the first start, give it a few minutes.
• If it stays at 0: UDP 443 is the usual cause, see UDP and QUIC. Then TCP 443, which must be open too, and the certificate: one dassie init has not obtained yet, or a temporary one the node made, is not taken. See Certificates, which also covers a domain name that does not point at this server.
• If some list it and others don't, the others may still be checking, or be down. The node registers again by itself, more slowly after each failure.
• Registration is free, so bootstrap nodes take only so many new nodes at a time, and a few from each network. Several nodes on one small network may have to wait their turn.

A periphery node registers nowhere, by design: its status says a periphery node instead.

Why it works this way: how bootstrap nodes check and list a node.

No peers

Your node is a member, but dassie status says peers: 0. Nodes peer only where they can settle, and they settle only through a settlement hub they share. So:

1. Is your node in a hub? dassie status has a Hub line for each. With none, join the testnet's XRP hub, which boot1 to boot4, four of the bootstrap nodes, keep: in the wallet, add XRP in a settlement hub on the XRP Ledger Testnet (Your hubs), or run dassie setup, which joins it unless you tell it otherwise (Choosing hubs). Your peers must be in the same hub.

2. Does the node follow the hub? A node offers to peer over a hub only once it knows where its account there stands. The Hub line should say in sync, or waiting for the first deposit to open it, and not connecting or failed.

3. Wait a little. Other nodes add yours to their tables only once they see a majority list it, and may refuse to peer until then. Your node asks again by itself.

Peers and the hubs they settle through are explained in Peers and holds and Settlement hubs.

Why it works this way: peers settle only over hubs.

The relay is not connected

dassie status says the relay is not connected, so Open Payments clients cannot reach the node through it. Right after a start the node says connecting, which is fine; this hint comes only once it has failed to reach the relay for a while. Payments over Interledger don't need the relay, and the node can still send.

The relay is the built-in front door, which the line names as … at https://relay.dassiepay.net, unless DASSIE_RELAY names another.

• Check that the server can reach the relay. The node connects over UDP to the port in the relay's URL. Where UDP is blocked, a node without a domain name tries a WebSocket over TCP to the same port instead, through a proxy if the network sets one (Behind a proxy). A node with a domain name uses UDP only.
• For the built-in front door: if the server reaches the internet and the line still says not connected, the front door may be down. Then there is nothing to fix on your side. The node dials again by itself, about once a minute, and connects once the front door is back. If it stays this way, update the node (Update now). The front door's key is built into each release, so a node keeps dialing an old key until a newer release brings the new one.
• For a relay you named: check that DASSIE_RELAY is exactly the entry the front door logs, with all four fields: the node ID, the URL and both keys. An entry whose ID does not match its keys stops the node at start, with a message naming DASSIE_RELAY (The node does not start). An entry that is whole but names another node than the relay at that URL fails to connect. Check, too, that the relay runs 0.3.0 or later. Nodes speak peer protocol dassie/2, which 0.2 releases don't.
• A DASSIE_RELAY that names your own node keeps no relay connection, and the log says so.

See A relay for how to name another relay, or none.

A ledger that won't sync

A ledger that stays at connecting, or says failed, cannot reach its server, or reaches the wrong one. The node connects out to the XRP Ledger's server over a secure WebSocket, on port 443: check that outgoing connections are allowed. A failed line says why, such as a server on the wrong network. To use another server, set the account's config with dassie setup, which reads your phrase from a file on the server:

$sudo dassie setup --passphrase-file /etc/dassie/phrase \
    --asset 'xrpl-testnet-account/0={"server":"wss://xrpl.example.net"}'
$sudo systemctl restart dassie

An account the node already uses takes the new server when the node next starts, hence the restart. The server must serve the proofs the node checks. More in Headless setup.

Still stuck?

Ask. Paste the output of dassie status, which names no secret, and the lines of journalctl -u dassie around the problem. The node never logs its setup link, and never sees your phrase. Get help says where to ask. Dassie is early, and every question tells us what to make clearer.