Skip to content

Commit

Permalink
feat!: Cryptographic verification of equivocation (#1287)
Browse files Browse the repository at this point in the history
* feat!: add ICS misbehaviour handling (#826)

* define msg to submit misbehaviour to provider

implement msg handling logic

e2e test msg handling logic

* wip: get byzantine validators in misbehavioiur handling

* add tx handler

* format HandleConsumerMisbehaviour

* add tx handler

* add debugging stuff

* Add misbehaviour handler

* create message for consumer double voting evidence

* add DRAFT double vote handler

* Add cli cmd for submit consumer double voting

* Add double-vote handler

* add last update

* fix jailing

* pass first jailing integration test

* format tests

* doc

* save

* update e2e tests'

* fix typo and improve docs

* remove unwanted tm evidence protofile

* fix typos

* update submit-consumer-misbehaviour cli description

* check that header1 and header2 have the same TrustedValidators

* feat: add e2e tests for ICS misbehaviour (#1118)

* remove unwanted changes

* fix hermes config with assigned key

* revert unwanted changes

* revert local setup

* remove log file

* typo

* update doc

* update ICS misbehaviour test

* update ICS misbehaviour test

* revert mixed commits

* add doc

* lint

* update to handle only equivocations

* improve doc

* update doc

* update E2E tests comment

* optimize signatures check

* doc

* update e2e tests

* linter

* remove todo

* Feat: avoid race condition in ICS misbehaviour handling (#1148)

* remove unwanted changes

* fix hermes config with assigned key

* revert unwanted changes

* revert local setup

* remove log file

* typo

* update doc

* update ICS misbehaviour test

* update ICS misbehaviour test

* revert mixed commits

* update ICS misbehaviour test

* update ICS misbehaviour test

* Add test for MsgSubmitConsumerMisbehaviour parsing

* fix linter

* save progress

* add CheckMisbehaviourAndUpdateState

* update integration tests

* typo

* remove e2e tests from another PRs

* cleaning'

* Update x/ccv/provider/keeper/misbehaviour.go

Co-authored-by: Anca Zamfir <[email protected]>

* Update x/ccv/provider/keeper/misbehaviour.go

Co-authored-by: Anca Zamfir <[email protected]>

* update integration tests

* save

* save

* nits

* remove todo

* lint

* Update x/ccv/provider/keeper/misbehaviour.go

---------

Co-authored-by: Anca Zamfir <[email protected]>
Co-authored-by: Marius Poke <[email protected]>

* Update x/ccv/provider/client/cli/tx.go

Co-authored-by: Anca Zamfir <[email protected]>

* Update x/ccv/provider/client/cli/tx.go

Co-authored-by: Anca Zamfir <[email protected]>

* add attributes to EventTypeSubmitConsumerMisbehaviour

* Update x/ccv/provider/keeper/misbehaviour.go

Co-authored-by: Anca Zamfir <[email protected]>

* Update x/ccv/provider/keeper/misbehaviour.go

Co-authored-by: Anca Zamfir <[email protected]>

* apply review suggestions

* fix docstring

* Update x/ccv/provider/keeper/misbehaviour.go

Co-authored-by: Anca Zamfir <[email protected]>

* fix link

* apply review suggestions

* update docstring

---------

Co-authored-by: Anca Zamfir <[email protected]>
Co-authored-by: Marius Poke <[email protected]>

* feat: improve ICS misbehaviour E2E testing coverage (#1225)

* update e2e tests

* update the chain halt assertion

* refactor: address comments of ICS Misbehaviour PRs #826 and #1148  (#1223)

* remove interface

* improve comment

* update godoc

* address last comments

* feat: add handler for consumer double voting (#1232)

* create new endpoint for consumer double voting

* add first draft handling logic

* first iteration of double voting

* draft first mem test

* error handling

* refactor

* add unit test of double voting verification

* remove evidence age checks

* document

* doc

* protogen

* reformat double voting handling

* logger nit

* nits

* check evidence age duration

* move verify double voting evidence to ut

* fix nit

* nits

* fix e2e tests

* improve double vote testing coverage

* remove TODO

* lint

* add UT for JailAndTombstoneValidator

* nits

* nits

* remove tombstoning and evidence age check

* lint

* typo

* improve godoc

* fix: tiny bug in `NewSubmitConsumerDoubleVotingCmd` (#1247)

* fix double voting cli

* fix bug double signing handler

* godoc

* nits

* revert wrong push of lasts commits

* fix: make `HandleConsumerDoubleVoting` works with provider pubkeys (#1254)

* fix double voting cli

* fix bug double signing handler

* godoc

* nits

* lint

* nit

* fix: verify equivocation using validator pubkey in `SubmitConsumerDoubleVoting` msg (#1264)

* verify dv evidence using malicious validator pubkey in infraction block header

* nits

* nits

* refactor: update the E2E tests to work with Hermes relayer v1.6.0 (#1278)

* save changes

* fix hermes config

* fist successful run

* nit

* nits

* nits

* doc and nits

* lint

* test: add E2E tests for double voting evidence handling (#1256)

* fix double voting cli

* add double-signing e2e test

* refortmat e2e double voting test

* godoc, revert unwanted changes

* nit

* verify dv evidence using malicious validator pubkey in infraction block header

* save changes

* fix hermes config

* fist successful run

* nit

* nits

* nits

* doc and nits

* lint

* refactor

* typo

* change hermes docker image

* nits

* Update tests/e2e/steps.go

Co-authored-by: Philip Offtermatt <[email protected]>

* address PR comments

* nits

---------

Co-authored-by: Philip Offtermatt <[email protected]>

* save

* fix nits

* update changelog and fix nits

---------

Co-authored-by: Simon Noetzlin <[email protected]>
Co-authored-by: Anca Zamfir <[email protected]>
Co-authored-by: Philip Offtermatt <[email protected]>
  • Loading branch information
4 people authored Sep 14, 2023
1 parent 494c4d2 commit b1a6f31
Show file tree
Hide file tree
Showing 36 changed files with 3,581 additions and 81 deletions.
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,13 @@ Add an entry to the unreleased section whenever merging a PR to main that is not
## v2.1.0-lsm-provider

* (feature!) [#1280](https://github.com/cosmos/interchain-security/pull/1280) provider proposal for changing reward denoms
* (feature!) [#826](https://github.com/cosmos/interchain-security/pull/826) add new endpoint to provider to handle consumer light client attacks
* (feature!) [#1227](https://github.com/cosmos/interchain-security/pull/1227) add new endpoint to provider to handle consumer double signing attacks


### Cryptographic verification of equivocation
* New feature enabling the provider chain to verify equivocation evidence on its own instead of trusting consumer chains, see [EPIC](https://github.com/cosmos/interchain-security/issues/732).


## v2.0.0-lsm

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ RUN go mod tidy
RUN make install

# Get Hermes build
FROM ghcr.io/informalsystems/hermes:1.4.1 AS hermes-builder
FROM otacrew/hermes-ics:evidence-cmd AS hermes-builder

# Get CometMock
FROM informalofftermatt/cometmock:latest as cometmock-builder
Expand Down
16 changes: 16 additions & 0 deletions docs/docs/features/slashing.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,3 +34,19 @@ The offending validator will effectively get slashed and tombstoned on all consu

<!-- markdown-link-check-disable-next-line -->
You can find instructions on creating `EquivocationProposal`s [here](./proposals#equivocationproposal).

# Cryptographic verification of equivocation
The Cryptographic verification of equivocation allows external agents to submit evidences of light client and double signing attack observed on a consumer chain. When a valid evidence is received, the malicious validators will be permanently jailed on the provider.

The feature is outlined in this [ADR-005](../adrs/adr-005-cryptographic-equivocation-verification.md)

By sending a `MsgSubmitConsumerMisbehaviour` or a `MsgSubmitConsumerDoubleVoting` transaction, the provider will
verify the reported equivocation and, if successful, jail the malicious validator.

:::info
Note that this feature can only lead to the jailing of the validators responsible for an attack on a consumer chain. However, an [equivocation proposal](#double-signing-equivocation) can still be submitted to execute the slashing and the tombstoning of the a malicious validator afterwards.
:::




35 changes: 35 additions & 0 deletions proto/interchain_security/ccv/provider/v1/tx.proto
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,15 @@ import "google/api/annotations.proto";
import "gogoproto/gogo.proto";
import "cosmos_proto/cosmos.proto";
import "google/protobuf/any.proto";
import "ibc/lightclients/tendermint/v1/tendermint.proto";
import "tendermint/types/evidence.proto";


// Msg defines the Msg service.
service Msg {
rpc AssignConsumerKey(MsgAssignConsumerKey) returns (MsgAssignConsumerKeyResponse);
rpc SubmitConsumerMisbehaviour(MsgSubmitConsumerMisbehaviour) returns (MsgSubmitConsumerMisbehaviourResponse);
rpc SubmitConsumerDoubleVoting(MsgSubmitConsumerDoubleVoting) returns (MsgSubmitConsumerDoubleVotingResponse);
}

message MsgAssignConsumerKey {
Expand All @@ -28,3 +33,33 @@ message MsgAssignConsumerKey {
}

message MsgAssignConsumerKeyResponse {}


// MsgSubmitConsumerMisbehaviour defines a message that reports a light client attack,
// also known as a misbehaviour, observed on a consumer chain
message MsgSubmitConsumerMisbehaviour {
option (gogoproto.equal) = false;
option (gogoproto.goproto_getters) = false;
string submitter = 1;
// The Misbehaviour of the consumer chain wrapping
// two conflicting IBC headers
ibc.lightclients.tendermint.v1.Misbehaviour misbehaviour = 2;
}

message MsgSubmitConsumerMisbehaviourResponse {}


// MsgSubmitConsumerDoubleVoting defines a message that reports
// a double signing infraction observed on a consumer chain
message MsgSubmitConsumerDoubleVoting {
option (gogoproto.equal) = false;
option (gogoproto.goproto_getters) = false;
string submitter = 1;
// The equivocation of the consumer chain wrapping
// an evidence of a validator that signed two conflicting votes
tendermint.types.DuplicateVoteEvidence duplicate_vote_evidence = 2;
// The light client header of the infraction block
ibc.lightclients.tendermint.v1.Header infraction_block_header = 3;
}

message MsgSubmitConsumerDoubleVotingResponse {}
52 changes: 45 additions & 7 deletions tests/e2e/actions.go
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ type StartChainAction struct {
validators []StartChainValidator
// Genesis changes specific to this action, appended to genesis changes defined in chain config
genesisChanges string
skipGentx bool
isConsumer bool
}

type StartChainValidator struct {
Expand Down Expand Up @@ -133,7 +133,7 @@ func (tr TestRun) startChain(
cmd := exec.Command("docker", "exec", tr.containerConfig.instanceName, "/bin/bash",
"/testnet-scripts/start-chain.sh", chainConfig.binaryName, string(vals),
string(chainConfig.chainId), chainConfig.ipPrefix, genesisChanges,
fmt.Sprint(action.skipGentx),
fmt.Sprint(action.isConsumer),
// override config/config.toml for each node on chain
// usually timeout_commit and peer_gossip_sleep_duration are changed to vary the test run duration
// lower timeout_commit means the blocks are produced faster making the test run shorter
Expand Down Expand Up @@ -170,6 +170,7 @@ func (tr TestRun) startChain(
tr.addChainToRelayer(addChainToRelayerAction{
chain: action.chain,
validator: action.validators[0].id,
consumer: action.isConsumer,
}, verbose)
}

Expand Down Expand Up @@ -280,6 +281,8 @@ func (tr TestRun) submitConsumerAdditionProposal(
if err != nil {
log.Fatal(err, "\n", string(bz))
}

tr.waitBlocks(action.chain, 1, 5*time.Second)
}

type submitConsumerRemovalProposalAction struct {
Expand Down Expand Up @@ -565,7 +568,7 @@ func (tr TestRun) startConsumerChain(
chain: action.consumerChain,
validators: action.validators,
genesisChanges: consumerGenesis,
skipGentx: true,
isConsumer: true,
}, verbose)
}

Expand Down Expand Up @@ -699,6 +702,7 @@ func (tr TestRun) startChangeover(
type addChainToRelayerAction struct {
chain chainID
validator validatorID
consumer bool
}

const hermesChainConfigTemplate = `
Expand All @@ -715,7 +719,8 @@ rpc_addr = "%s"
rpc_timeout = "10s"
store_prefix = "ibc"
trusting_period = "14days"
websocket_addr = "%s"
event_source = { mode = "push", url = "%s", batch_delay = "50ms" }
ccv_consumer_chain = %v
[chains.gas_price]
denom = "stake"
Expand Down Expand Up @@ -814,7 +819,7 @@ func (tr TestRun) addChainToHermes(
keyName,
rpcAddr,
wsAddr,
// action.consumer,
action.consumer,
)

bashCommand := fmt.Sprintf(`echo '%s' >> %s`, chainConfig, "/root/.hermes/config.toml")
Expand All @@ -828,7 +833,15 @@ func (tr TestRun) addChainToHermes(
}

// Save mnemonic to file within container
saveMnemonicCommand := fmt.Sprintf(`echo '%s' > %s`, tr.validatorConfigs[action.validator].mnemonic, "/root/.hermes/mnemonic.txt")
var mnemonic string
if tr.validatorConfigs[action.validator].useConsumerKey && action.consumer {
mnemonic = tr.validatorConfigs[action.validator].consumerMnemonic
} else {
mnemonic = tr.validatorConfigs[action.validator].mnemonic
}

saveMnemonicCommand := fmt.Sprintf(`echo '%s' > %s`, mnemonic, "/root/.hermes/mnemonic.txt")
fmt.Println("Add to hermes", action.validator)
//#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments.
bz, err = exec.Command("docker", "exec", tr.containerConfig.instanceName, "bash", "-c",
saveMnemonicCommand,
Expand Down Expand Up @@ -1716,7 +1729,7 @@ func (tr TestRun) submitChangeRewardDenomsProposal(action submitChangeRewardDeno
//#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments.
// CHANGE REWARDS DENOM PROPOSAL
bz, err = exec.Command("docker", "exec", tr.containerConfig.instanceName, providerChain.binaryName,
"tx", "gov", "submit-legacy-proposal", "change-reward-denoms", "/change-reward-denoms-proposal.json",
"tx", "gov", "submit-proposal", "change-reward-denoms", "/change-reward-denoms-proposal.json",
`--from`, `validator`+fmt.Sprint(action.from),
`--chain-id`, string(providerChain.chainId),
`--home`, tr.getValidatorHome(providerChain.chainId, action.from),
Expand Down Expand Up @@ -1864,6 +1877,8 @@ func (tr TestRun) assignConsumerPubKey(action assignConsumerPubKeyAction, verbos
valCfg.useConsumerKey = true
tr.validatorConfigs[action.validator] = valCfg
}

time.Sleep(1 * time.Second)
}

// slashThrottleDequeue polls slash queue sizes until nextQueueSize is achieved
Expand Down Expand Up @@ -1925,3 +1940,26 @@ func (tr TestRun) GetPathNameForGorelayer(chainA, chainB chainID) string {

return pathName
}

// Run an instance of the Hermes relayer using the "evidence" command,
// which detects evidences committed to the blocks of a consumer chain.
// Each infraction detected is reported to the provider chain using
// either a SubmitConsumerDoubleVoting or a SubmitConsumerMisbehaviour message.
type startConsumerEvidenceDetectorAction struct {
chain chainID
}

func (tr TestRun) startConsumerEvidenceDetector(
action startConsumerEvidenceDetectorAction,
verbose bool,
) {
chainConfig := tr.chainConfigs[action.chain]
// run in detached mode so it will keep running in the background
//#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments.
bz, err := exec.Command("docker", "exec", "-d", tr.containerConfig.instanceName,
"hermes", "evidence", "--chain", string(chainConfig.chainId)).CombinedOutput()
if err != nil {
log.Fatal(err, "\n", string(bz))
}
tr.waitBlocks("provi", 10, 2*time.Minute)
}
119 changes: 119 additions & 0 deletions tests/e2e/actions_consumer_misbehaviour.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,119 @@
package main

import (
"bufio"
"fmt"
"log"
"os/exec"
"strconv"
"time"
)

// forkConsumerChainAction forks the consumer chain by cloning of a validator node
// Note that the chain fork is running in an different network
type forkConsumerChainAction struct {
consumerChain chainID
providerChain chainID
validator validatorID
relayerConfig string
}

func (tr TestRun) forkConsumerChain(action forkConsumerChainAction, verbose bool) {
valCfg := tr.validatorConfigs[action.validator]

//#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments.
configureNodeCmd := exec.Command("docker", "exec", tr.containerConfig.instanceName, "/bin/bash",
"/testnet-scripts/fork-consumer.sh", tr.chainConfigs[action.consumerChain].binaryName,
string(action.validator), string(action.consumerChain),
tr.chainConfigs[action.consumerChain].ipPrefix,
tr.chainConfigs[action.providerChain].ipPrefix,
valCfg.mnemonic,
action.relayerConfig,
)

if verbose {
log.Println("forkConsumerChain - reconfigure node cmd:", configureNodeCmd.String())
}

cmdReader, err := configureNodeCmd.StdoutPipe()
if err != nil {
log.Fatal(err)
}
configureNodeCmd.Stderr = configureNodeCmd.Stdout

if err := configureNodeCmd.Start(); err != nil {
log.Fatal(err)
}

scanner := bufio.NewScanner(cmdReader)

for scanner.Scan() {
out := scanner.Text()
if verbose {
log.Println("fork consumer validator : " + out)
}
if out == done {
break
}
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}

time.Sleep(5 * time.Second)
}

type updateLightClientAction struct {
chain chainID
hostChain chainID
relayerConfig string
clientID string
}

func (tr TestRun) updateLightClient(
action updateLightClientAction,
verbose bool,
) {
// retrieve a trusted height of the consumer light client
trustedHeight := tr.getTrustedHeight(action.hostChain, action.clientID, 2)

//#nosec G204 -- Bypass linter warning for spawning subprocess with cmd arguments.
cmd := exec.Command("docker", "exec", tr.containerConfig.instanceName, "hermes",
"--config", action.relayerConfig,
"update",
"client",
"--client", action.clientID,
"--host-chain", string(action.hostChain),
"--trusted-height", strconv.Itoa(int(trustedHeight.RevisionHeight)),
)
if verbose {
log.Println("updateLightClientAction cmd:", cmd.String())
}

bz, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err, "\n", string(bz))
}

tr.waitBlocks(action.hostChain, 5, 30*time.Second)
}

type assertChainIsHaltedAction struct {
chain chainID
}

// assertChainIsHalted verifies that the chain isn't producing blocks
// by checking that the block height is still the same after 20 seconds
func (tr TestRun) assertChainIsHalted(
action assertChainIsHaltedAction,
verbose bool,
) {
blockHeight := tr.getBlockHeight(action.chain)
time.Sleep(20 * time.Second)
if blockHeight != tr.getBlockHeight(action.chain) {
panic(fmt.Sprintf("chain %v isn't expected to produce blocks", action.chain))
}
if verbose {
log.Printf("assertChainIsHalted - chain %v was successfully halted\n", action.chain)
}
}
Loading

0 comments on commit b1a6f31

Please sign in to comment.