Red Queen's: New Sync Proposal

As part of Ethereum 1x efforts, we want to tackle the issue of the growing blockchain state. Large state size leads to long sync times. On top of that, geth’s fast sync is not that fast and takes a few hours. Parity’s warp sync is faster, but is not supported in geth. We propose a new sync protocol and an algorithm, which we believe can perform snapshot sync in less than an hour. The idea is somewhat similar to Warp Sync and especially to Leaf Sync (under development in geth). Moreover, the new protocol caters for clients with a different storage structure than the canonical Patricia trie (e.g. turbo-geth).

The new protocol and algorithm are described in a paper. See also my presentation at Ethereum Core Devs Eth1x/Istanbul Planning Meeting in Berlin.

This is the first version; comments, suggestions, critique are most welcome.

4 Likes

The new protocol is related to Forming a Ring: ETH v64 Wire Protocol Ring.

1 Like

See also a gitter discussion about new sync protocols.

An update: we’re going to use “Red Queen” for a sync algorithm (one out of a few possible) and “Firehose” for the new sync protocol. A new version 0.3 of Firehose is available at https://github.com/ledgerwatch/turbo-geth/blob/red-queen/docs/firehose.MD.