Skip to content

Nimbus Roadmap

Jump to bottom
zah edited this page Mar 8, 2018 · 17 revisions

This document attempts to specify a list of high-level components of the Nimbus project that can be developed and tested in isolation. It tries to identify the dependencies between the development tasks, so the implementation efforts can be scheduled in parallel where appropriate. Please keep in mind that the list is not exhaustive and further planning is necessary before all tasks are specified in the same level of detail.

Development tasks:

1. Select an AES implementation

The AES library should support both AES256 in CTR mode (as per the RLPx spec) and GCM mode (used in Whisper v6)

Suggested library: Crypto++

2. Implement ECIES

Required for the encyprted handshake. Based on libsecp256k1.

Reference implementation:

Depends on:

3. Select Scrypt library (optional)

Both cpp-ethereum and Parity also use Scrypt in their local keystore code. It doesn't seem mandatory for the implementation.

4. Ethereum Bloom Filter

Bloom filters are used for efficient monitoring of the transaction logs/events, stored in the receipt trie of the blockchain, for topic selection in Whisper and some other places.

Reference implementation:

Existing Nim library:

5. Ethereum key file

The key file stores the keys associated with one or more accounts in password-protected encrypted form.

Reference implementation:

Sub-tasks:

  • Safe storage
  • Key generation / passphrase recovery
  • Use native APIs on mobile platforms

Depends on:

6. JSON-RPC Server

Development already started. This will be an ongoing effort and the list of supported calls should increase as other parts of the system are developed.

Reference information:

Test suite

Sub-tasks:

  • CI tests for all libraries
  • Validate that the entire history of Ethereum can be executed by Nimbus
  • Check that we fit in a certain memory/compute budget?

7. Node discovery

This uses a modified and stripped down version of Kademlia. Kademlia us also used in Swarm for resource discovery and some of the logic for maintaining peer connection tables can be shared between the two implementations.

In the sharded blockchan, each shard is represented by a separate P2P network. This is implemented by running the same node discovery protocol, but with a different peer connection port:

In the future, Ethereum may support alternative transport protocols (such as lib2p2). To enable this, there is new version 5 of the node discovery protocol in development, which allow the nodes to advertise alternative connection methods (ENR):

Sub-tasks:

8. UPnP support

The standard Ethereum protocols use TCP to communicate between peers. Hosts with private IP addresses must set up port redirection on their routers. UPnP can be used to automate this.

9. Authenticated handshake

https://github.com/ethereum/devp2p/blob/master/rlpx.md#encrypted-handshake

Sub-tasks:

  • Authenticate previously known nodes with an old session key
  • Authenticate newly discovered nodes
  • Support RLPx sub-protocol negotiation in the authentication message (capabilities)
  • Derive shared secrets from handshake

Depends on:

10. RLPx sub-protocols

The Ethereum wire protocols are transport-agnostic. Our framework should allow running them over the TCP connections established after node discovery or through connections established with libp2p or uTP in the future. It should also be possible to define unit tests involving multiple agents simulated within a single process (bypassing the network stack).

Issues:

None of the protocols currently supports using NAT traversal for directly establishing P2P connections. This may be important for supporting Video and Audio calls in Status. We can solve this by establishing such connections in an out-of-band protocol (e.g. by linking WebRTC or something similar) or by contributing EIPs for the existing protocols.

Sub-tasks:

  • SubProtocol registry

  • Active protocol map (shared objects)

  • Message dispatcher

  • Subprotocol definition DSL

  • Protocol negotiation

  • Per-subprotocol send queue

  • Chunking

  • Peer reputation tracking (with persistence)

  • Flow control (optional)

  • Pre-sending PoW priority increase

11. The new binary trie

Replaces the old hexary trie in the new blockchain shards.

Sub-tasks:

  • Add support for partial tries
  • Support for obtaining and verifying Merkle proofs

Reference implementations:

12. New parametric Blockchain definition

There should be sufficient parametrization to allow:

The new shard chain uses a new account creation method:

... which requires 3 new opcodes to be added:

  • SIGHASH
  • PAYGAS
  • CREATE2

Current outstanding tasks:

  • Stateless and stateful execution with roll-back
  • Maintenance of a receipt trie
  • Block validation
  • Precompiled contracts
  • Transaction pool

Depends on:

Extra tasks:

  • VM execution tracing

13. Blockchain Sync Algorithms

Fast sync uses the standard Ethereum sub-protocol.

Parity's warp sync is based on a custom server and protocol developed by the Parity team.

The sharding client features an even faster stateless sync protocol that aims to obtain only the latest block headers and to perform a randomized verification that the blockchain data is available upon request:

Depends on:

14. Known contract abstraction / DApp framework

Make it easier to interact with known contracts such as Casper, the Sharding VMC, etc. Each such contract has a well-known address and a particular API that can be expressed in standard Nim.

Libraries such as web3.js and web.py offer similar functionality.

Sub-tasks:

  • Nim-types-to-ABI-types translation and contract FFI DSL.
  • Transaction creation logic
  • Event monitoring logic
  • Provide high-level information about the contract for the UI layer (NatSpec, etc)

Depends on:

15. Light Clients

The light client operates without storing the full chain locally. Most operations depends on Asynchronous APIs that will downloads parts of the chain on-demand. Light clients can also execute DApps by monitoring the new blocks for events of interest.

Sub-tasks:

  • Events monitoring
  • State-fetching APIs
  • Flow control
  • Fetch queue

Depends on:

16. Stateless Clients

Sub-tasks:

  • Transaction state witness extraction
  • Stateless sync
  • State objects and "pure" VM execution
  • Track updates to all keys in the witness in an in-memory store.

Depends on:

17. Casper daemon

Sub-tasks:

  • Implement the new Fork-choice rule
  • Monitor epochs, send prepares and commits when appropriate.
  • Avoid triggering the slashing conditions by maintaing a local database of past actions.
  • Provide an UI for adding deposits, tracking revenue and withdrawing funds.

Depends on:

18. Sharding daemon

PY-EVM Implementation Roadmap:

Sub-tasks:

  • Shard monitoring/switching
  • Block/Collation creation
  • New account creation logic
  • New VM opcodes
  • Provide an UI for adding deposits, tracking revenue and withdrawing funds.

Depends on:

  • Binary Trie, Overlayed/Tracked DB back-end. (used to keep track of modifications created by a transaction and to model the transaction Witness data)

  • Stateless Clients

  • Parametric chain implementation

  • Known contract abstraction

19. Etehreum name service resolver

Sub-tasks:

  • Resolve names by recursive ENS contract application

Depends on:

20. Swarm

Sub-tasks:

  • Kademlia search
  • File chunks upload / download manager
  • Hierarchical hash verification
  • Virtual file system based on manifests
  • PSS (messaging over swarm)
  • NAT Traversal brokerage over PSS (e.g. for WebRTC)
  • Light mode of operation, relay nodes
  • SWAP, SWEAR, SWINDLE, chequebooks

Depends on:

21. Whisper

While Swarm seems a better fit for Status's needs, we may also implement the Whisper protocol, which is currently at version 6:

Overview information (previous versions):

Sub-tasks:

  • Envelope encryption
  • Mail Server
  • Filter table
  • Onion routing (Darkness)

Depends on:

22. eWASM VM

Sub-tasks:

  • Integrate Hera
  • Extract all extern methods in a module shared between the VMs

Depends on:

  • Parametric chain

23. Integrate libsnark

Developers can take advantage of the homomorphic encryption and the zero-knowledge proofs available in libsnark to implement currencies and other smart contracts with stronger privacy (similar to zcash). Ethereum supports this by offering the operations from libsnark as precompiled contracts.

Depends on:

  • Precompiled contracts

24. Support IPFS

No concrete plans yet.

25. Command-line interface, Configuration, Local directory structure, Build Targets, Web Interface

Sub-tasks:

  • Provide options for launching the various daemons and modes of Nimbus
  • Define a common scheme for specifying parameters (as command-line switches, as env vars, in configuration files, etc)
  • Provide abstractions over local file system usage (e.g. XDG, AppData, Android and iOS specific APIs)
  • Provide compile-time defines for turning off features of Nimbus (i.e. allow the lite clients to omit most of the code)
  • Test example applications using Nimbus as a library
  • Define various UIs for Nimbus (Web UI, GUI, etc)
  • Android build / CI test suite
  • iOS build / CI test suite

26. Nim smart contract development

Define a Nim DSL for creating smart contracts.

Sub-tasks:

  • Support creating library contracts
  • Compile Nim to Solidity ASM or Vyper LLL
  • Compile Nim to eWASM (either through the C back-end or through LLVM)
  • Enhance security through static code analysis / formal methods

27. Fuzzing framework

Test the implementation of all building block libraries such as RLP, the Tries, the VM, etc through a fuzzer such as afl-fuzz. Teach the fuzzer about Nim.

Clone this wiki locally