Skip to content

Latest commit

 

History

History
298 lines (236 loc) · 10.6 KB

04-upstream-config.adoc

File metadata and controls

298 lines (236 loc) · 10.6 KB
Raw

Upstreams Configuration

Dshackle can connect to multiple independent APIs ("upstreams") and provides an unified API on top of it.

Supported upstream protocols:

  • JSON RPC

  • Websockets

  • gRPC (i.e. can connect to another Dshackle)

Those protocols can be configures with additional security, TLS and authentication.

Notes on upstream configuration

Ethereum

  • Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC. If it’s available on your node, it’s suggested to configure both JSON RPC and WebSocket connection

Bitcoin

  • Bitcoind needs to be configured to index/track addresses that you’re going to request. Make sure you configure it to index your addresses with importaddress. If you request balance for an address that is not indexed then it returns 0 balance.

  • To track all transactions you need to setup index for transactions, which is disabled by default. Run it with -reindex option, or set txindex=1 in the config.

Example Configuration

upstreams.yaml
version: v1

cluster:
  defaults:
  - blockchains:
      - ethereum
    min-peers: 10
  - blockchains:
      - kovan
    min-peers: 2
  upstreams:
    - id: us-nodes
      blockchain: auto
      priority: 10
      connection:
        dshackle:
          url: https://35.226.252.117
          tls:
            ca: ca.crt
            certificate: client.crt
            key: client.p8.key
    - id: infura-eth
      blockchain: ethereum
      priority: 5
      role: fallback
      labels:
        provider: infura
      disable-validation: true
      connection:
        ethereum:
          rpc:
            url: "https://mainnet.infura.io/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}
          ws:
            url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}

There are two main segments for upstreams configuration:

  • upstreams - a list of API to connect to, with all configuration specific for upstream and chain

  • and default options as common configuration options applied to all nodes in that group

In the example above we have:

  • default configuration for Ethereum Mainnet which accepts upstream as valid when it not in fast synchronization mode and has at least 10 peers.For Kovan Testnet nodes the requirements are much relieved

  • as upstreams it has 2 configurations

    • balancer connects to another Dshackle/another machine by using gRPC protocol

      • accepts (i.e. proxies) any blockchain available on that remote

      • verifies TLS certificate of the server

      • uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a certificate

    • connects to Infura provided Ethereum Mainnet

      • as a fallback upstream, which means that it’s used only if us-nodes fails

      • configuration is using placeholders for ${INFURA_USER} and ${INFURA_PASSWD} which will be replaced with corresponding environment variables values

      • uses Basic Authentication to authenticate requests on Infura

      • label [provider: infura] is set for that particular upstream, which can be selected during a request.For example for some requests you may want to use nodes with that label only, i.e. "send that tx to infura nodes only", or "read only from archive node, with label [archive: true]"

      • upstream validation (peers, sync status, etc) is disabled for that particular upstream

    • upstream us-nodes has a higher priority than infura-eth so if there is a different block on the same height (i.e., a fork) then Dshackle chooses block from us-nodes. Note that it is specific only for Proof-of-Stake blockchains, such as Ethereum.

Fork Choice configuration

When two different upstream produce a different block it’s critical to route the request traffic to an upstream on the right fork.

With Proof-of-Work blockchains Dshackle chooses the block with more works produced, and that’s the general consensus for PoW blockchains.

For Proof-of-Stake this is impossible and instead of Total Work you have to configure a priority of upstream, i.e., the process is more manual. In a situation when Dshackle sees a fork it chooses the upstreams with the block on the side with the highest priority.

Configuration option for priorities is priority:
- id: eth-one
  blockchain: ethereum
  priority: 10

- id: eth-two
  blockchain: ethereum
  priority: 20

- id: eth-three
  blockchain: ethereum
  priority: 15

In this case upstream eth-two has the highest priority, and it’s blocks are considered as true in case of a fork. If the upstream is down for a some reason, then blocks from eth-three are used instead.

Note
 Priority is a configuration for a Fork decision, and it doesn’t change the priority of the traffic. Except the situation when Dshackle sees a fork between different upstreams; in this case all traffic is routed only to upstreams on the right side of the fork.
Note
 Configuration applies only for Proof-of-Stack and for Proof-of-Work blockchains priority options is not used.

Roles and Fallback upstream

By default, the Dshackle connects to each upstream in a Round-Robin basis, i.e. sequentially one by one. If you need more gradual control over the order of which upstream is used and when you can assign following roles:

  • primary (default role if nothing specified)

  • secondary

  • fallback

Where primary and secondary are considered here a standard upstreams, and fallback is used on failure of standard upstreams. I.e. the Dshackle always starts with making requests to standard upstreams. If all of them failed, if responses are inconsistent (ex. for eth_getTransactionCount), or when it needs to broadcast to a wider network (sendrawtransaction), then upstreams with role fallback cames to use.

The internal request order is (goes to next only if all upstreams on current step a not available or failed):

  1. tries with primary upstreams

  2. tries with secondary upstream

  3. …​ delay (100ms at first, increased each iteration)

  4. tries with primary upstreams

  5. tries with secondary upstream

  6. tries with fallback upstreams

  7. …​ go to step 3

Steps 3-6 are repeated until a valid response received, or a timeout for the original request is reached.

In general: - you set role secondary for upstream in another cluster/datacenter - you set role fallback for an external upstream which may be provided by a third party, and you want to use it as a last resort

Configuration options

Options (default or as part of upstream config):

Option Default Description

disable-validation

false

if true then Dshackle will not try to verify status of the upstream (could be useful for a trusted cloud provider such as Infura, but disabling it is not recommended for a normal node)

min-peers

3

specify minimum amount of connected peers, Dshackle will not use upstream with less than specified number

timeout

60

timeout in seconds after which request to the upstream will be discarded (and may be retried on an another upstream)

balance

true for ethereum, false for bitcoin

specify if this node should be used to fetch balance for an address

Connection type

Dshackle currently supports

  • rpc a standard Ethereum JSON RPC

  • ws websocket connection (supposed to be used in addition to rpc connection)

  • dshackle connects to another Dshackle instance

Bitcoin Methods

By default, a bitcoin upstream allows call to the following JSON RPC methods:

  • getbestblockhash

  • getblock

  • getblocknumber

  • getblockcount

  • gettransaction

  • getrawmempool

  • getrawtransaction

  • gettxout

  • getreceivedbyaddress

  • listunspent

  • sendrawtransaction

Plus following methods are answered directly by Dshackle

  • getmemorypool

  • getconnectioncount

  • getnetworkinfo

Ethereum Methods

By default, an ethereum upstream allows calls to the following JSON RPC methods:

  • eth_gasPrice

  • eth_call

  • eth_estimateGas

  • eth_getBlockTransactionCountByHash

  • eth_getUncleCountByBlockHash

  • eth_getBlockByHash

  • eth_getTransactionByHash

  • eth_getTransactionByBlockHashAndIndex

  • eth_getStorageAt

  • eth_getCode

  • eth_getUncleByBlockHashAndIndex

  • eth_getTransactionCount

  • eth_blockNumber

  • eth_getBalance

  • eth_sendRawTransaction

  • eth_getBlockTransactionCountByNumber

  • eth_getUncleCountByBlockNumber

  • eth_getBlockByNumber

  • eth_getTransactionByBlockNumberAndIndex

  • eth_getTransactionReceipt

  • eth_getUncleByBlockNumberAndIndex

  • eth_feeHistory

  • eth_getLogs

Plus following methods are answered directly by Dshackle

  • net_version

  • net_peerCount

  • net_listening

  • web3_clientVersion

  • eth_protocolVersion

  • eth_syncing

  • eth_coinbase

  • eth_mining

  • eth_hashrate

  • eth_accounts

It’s possible to enable additional methods that are available on upstream, or disable an existing method. For that purpose there is methods configuration:

upstreams:
  - id: my-node
    blockchain: ethereum
    labels:
      archive: true
    methods:
      enabled:
        - name: trace_transaction
      disabled:
        - name: eth_getBlockByNumber

Such configuration option allows executing method trace_transaction and also disables eth_getBlockByNumber on that particular upstream. If a client requests to execute method trace_transaction then it will be scheduled to that upstream (or any upstream with such method enabled).

Note
 It’s especially useful when used together with upstream labels.If an archive upstream has label archive: true it’s possible to specify that the client wants to execute method trace_transaction only on an archive node(s), which has complete historical data for tracing.

Static Methods

You can overwrite existing methods or add new ones using a static response:

upstreams:
  - id: my-node
    blockchain: ethereum
    methods:
      enabled:
        - name: net_version
          static: "\"100000\""
        - name: eth_chainId
          static: "0x186a0"
        - name: eth_custom_array
          static: '["custom_array_response"]'
        - name: eth_custom_bool
          static: "false"

Authentication

TLS

All connection types can use TLS secured connection, with optional client certificate authentication:

  • ca path to certificate required from remote server

  • optional certificate and key for client authentication.

Note
 Please note that key must be encoded with PKCS 8

Basic Authentication

For JSON RPC and Websockets a Basic Authentication can be used:

  • username - username

  • password - password