Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tests(integration): port x/bank tests to server/v2 app #21912

Open
wants to merge 70 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 69 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
da84451
audit NewIntegrationApp usages
kocubinski Sep 3, 2024
eb347f6
Update v2-audit.md
kocubinski Sep 3, 2024
2e9835c
very basic port
kocubinski Sep 6, 2024
b0cc5a5
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 6, 2024
14686ba
merge fix
kocubinski Sep 6, 2024
74358ca
basic construction seems to be working
kocubinski Sep 10, 2024
ba91038
refactor: remove viper as a direct dependency
kocubinski Sep 10, 2024
c4d0834
remove unused interface
kocubinski Sep 10, 2024
a080b2e
not sure how this ended up so big
kocubinski Sep 11, 2024
f09a8fe
clean up
kocubinski Sep 11, 2024
ce51d9e
go mod tidy all
kocubinski Sep 11, 2024
441423d
lint fix
kocubinski Sep 11, 2024
7e508c3
fix simapp v1 build with core replace
kocubinski Sep 11, 2024
99c2a5c
add comment
kocubinski Sep 11, 2024
a6e0c1f
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/app-v2-…
kocubinski Sep 11, 2024
e0c9e3d
go mod tidy all
kocubinski Sep 11, 2024
8c4d79d
Merge branch 'kocu/app-v2-config' into kocu/v2-integration-tests
kocubinski Sep 11, 2024
2d28d64
wip
kocubinski Sep 11, 2024
0e71ce3
fix(runtime/v2): provide default factory options if unset in app builder
kocubinski Sep 12, 2024
3ad2a6e
fix simapp/v2 usage
kocubinski Sep 12, 2024
d8a6ed7
whitespace?
kocubinski Sep 12, 2024
de2a723
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 12, 2024
a47a286
tidy
kocubinski Sep 12, 2024
c8ea058
fix test
kocubinski Sep 12, 2024
2410c04
Merge branch 'kocu/default-factory-opts' into kocu/v2-integration-tests
kocubinski Sep 12, 2024
8d3698f
config fix
kocubinski Sep 12, 2024
57040e2
need comet service bindings
kocubinski Sep 13, 2024
995974c
looks like a valid construction
kocubinski Sep 13, 2024
69598d4
genesis service
kocubinski Sep 15, 2024
29b0ebe
genesis working
kocubinski Sep 15, 2024
a977338
refactor(stf): rm RunWithCtx
kocubinski Sep 15, 2024
30814ac
migrate export genesis and remove stf.RunWithCtx
kocubinski Sep 15, 2024
ef50b4f
revert line wrapping
kocubinski Sep 16, 2024
177a0da
small refactor move genesis services
kocubinski Sep 16, 2024
25f0244
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/rm-stf-…
kocubinski Sep 16, 2024
45ff83a
go mod tidy all
kocubinski Sep 16, 2024
16a0e99
revert vscode change
kocubinski Sep 16, 2024
8d1014e
Merge branch 'main' into kocu/rm-stf-run-with-ctx
kocubinski Sep 16, 2024
9fe1020
update genesis service
kocubinski Sep 16, 2024
80db042
clean up
kocubinski Sep 16, 2024
57ec239
fix typo
kocubinski Sep 16, 2024
c20649d
fix test
kocubinski Sep 16, 2024
ea930ac
fix test
kocubinski Sep 16, 2024
c508350
Merge branch 'kocu/rm-stf-run-with-ctx' into kocu/v2-integration-tests
kocubinski Sep 16, 2024
8b9b66c
fix merge badness
kocubinski Sep 16, 2024
e0eb0ef
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 20, 2024
b6d6f75
kinda working
kocubinski Sep 20, 2024
d8fc366
passin with lots of debug printlns
kocubinski Sep 23, 2024
00dd82a
working integration test!
kocubinski Sep 24, 2024
74e8eed
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 24, 2024
710a1aa
clean up
kocubinski Sep 24, 2024
ae35d88
port another test
kocubinski Sep 24, 2024
eb1f49e
port remaining bank tests
kocubinski Sep 25, 2024
e37ff14
revert whitespace changes
kocubinski Sep 25, 2024
7e6b4db
moar whitespace
kocubinski Sep 25, 2024
d3e474c
why did x/upgrade get touched?
kocubinski Sep 25, 2024
8292c98
godocs
kocubinski Sep 25, 2024
de81b74
Merge branch 'main' into kocu/v2-integration-tests
kocubinski Sep 25, 2024
706a591
more cleanup but stuck on gas service
kocubinski Sep 27, 2024
9e42ec8
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 27, 2024
db41984
tests passing
kocubinski Sep 27, 2024
869a492
rever gas addition to stf query return
kocubinski Sep 27, 2024
a8f7d8e
clean up
kocubinski Sep 27, 2024
6a9e803
revert whitespace
kocubinski Sep 27, 2024
9cf71f4
godocs
kocubinski Sep 30, 2024
47136de
runtime/v2 clean up
kocubinski Sep 30, 2024
52c502a
add comment
kocubinski Sep 30, 2024
8d34c75
clean up
kocubinski Sep 30, 2024
55e77a5
linting
kocubinski Sep 30, 2024
4c9de97
Merge branch 'main' of github.com:cosmos/cosmos-sdk into kocu/v2-inte…
kocubinski Sep 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions codec/depinject.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ import (
"github.com/cosmos/cosmos-sdk/codec/types"
)

var DefaultProviders = depinject.Provide(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shal we use those in app_di (v1 & v2) to simplify things?

ProvideInterfaceRegistry,
ProvideLegacyAmino,
ProvideProtoCodec,
ProvideAddressCodec,
)

func ProvideInterfaceRegistry(
addressCodec address.Codec,
validatorAddressCodec address.ValidatorAddressCodec,
Expand Down
1 change: 1 addition & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -184,6 +184,7 @@ require (
replace (
cosmossdk.io/api => ./api
cosmossdk.io/collections => ./collections
cosmossdk.io/core => ./core
cosmossdk.io/store => ./store
cosmossdk.io/x/bank => ./x/bank
cosmossdk.io/x/staking => ./x/staking
Expand Down
2 changes: 0 additions & 2 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@ buf.build/gen/go/cosmos/gogo-proto/protocolbuffers/go v1.34.2-20240130113600-88e
buf.build/gen/go/cosmos/gogo-proto/protocolbuffers/go v1.34.2-20240130113600-88ef6483f90f.2/go.mod h1:HqcXMSa5qnNuakaMUo+hWhF51mKbcrZxGl9Vp5EeJXc=
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cloud.google.com/go v0.34.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
cosmossdk.io/core v1.0.0-alpha.3 h1:pnxaYAas7llXgVz1lM7X6De74nWrhNKnB3yMKe4OUUA=
cosmossdk.io/core v1.0.0-alpha.3/go.mod h1:3u9cWq1FAVtiiCrDPpo4LhR+9V6k/ycSG4/Y/tREWCY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29 h1:NxxUo0GMJUbIuVg0R70e3cbn9eFTEuMr7ev1AFvypdY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29/go.mod h1:8s2tPeJtSiQuoyPmr2Ag7meikonISO4Fv4MoO8+ORrs=
cosmossdk.io/depinject v1.0.0 h1:dQaTu6+O6askNXO06+jyeUAnF2/ssKwrrszP9t5q050=
Expand Down
49 changes: 5 additions & 44 deletions runtime/v2/builder.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,28 +6,22 @@ import (
"errors"
"fmt"
"io"
"path/filepath"

"cosmossdk.io/core/appmodule"
appmodulev2 "cosmossdk.io/core/appmodule/v2"
"cosmossdk.io/core/server"
"cosmossdk.io/core/store"
"cosmossdk.io/core/transaction"
"cosmossdk.io/runtime/v2/services"
"cosmossdk.io/server/v2/appmanager"
"cosmossdk.io/server/v2/stf"
"cosmossdk.io/server/v2/stf/branch"
"cosmossdk.io/store/v2/db"
rootstore "cosmossdk.io/store/v2/root"
)

// AppBuilder is a type that is injected into a container by the runtime/v2 module
// (as *AppBuilder) which can be used to create an app which is compatible with
// the existing app.go initialization conventions.
type AppBuilder[T transaction.Tx] struct {
app *App[T]
config server.DynamicConfig
storeOptions *rootstore.Options
app *App[T]

// the following fields are used to overwrite the default
branch func(state store.ReaderMap) store.WriterMap
Expand Down Expand Up @@ -99,6 +93,10 @@ func (a *AppBuilder[T]) Build(opts ...AppBuilderOption[T]) (*App[T], error) {
}
}

if a.app.db == nil {
return nil, fmt.Errorf("app.db is not set, it is required to build the app")
}

if err := a.app.moduleManager.RegisterServices(a.app); err != nil {
return nil, err
}
Expand All @@ -122,37 +120,6 @@ func (a *AppBuilder[T]) Build(opts ...AppBuilderOption[T]) (*App[T], error) {
}
a.app.stf = stf

home := a.config.GetString(FlagHome)
scRawDb, err := db.NewDB(
db.DBType(a.config.GetString("store.app-db-backend")),
"application",
filepath.Join(home, "data"),
nil,
)
if err != nil {
panic(err)
}

var storeOptions rootstore.Options
if a.storeOptions != nil {
storeOptions = *a.storeOptions
} else {
storeOptions = rootstore.DefaultStoreOptions()
}
factoryOptions := &rootstore.FactoryOptions{
Logger: a.app.logger,
RootDir: home,
Options: storeOptions,
StoreKeys: append(a.app.storeKeys, "stf"),
SCRawDB: scRawDb,
}

rs, err := rootstore.CreateRootStore(factoryOptions)
if err != nil {
return nil, fmt.Errorf("failed to create root store: %w", err)
}
a.app.db = rs

appManagerBuilder := appmanager.Builder[T]{
STF: a.app.stf,
DB: a.app.db,
Expand Down Expand Up @@ -251,9 +218,3 @@ func AppBuilderWithPostTxExec[T transaction.Tx](postTxExec func(ctx context.Cont
a.postTxExec = postTxExec
}
}

func AppBuilderWithStoreOptions[T transaction.Tx](opts *rootstore.Options) AppBuilderOption[T] {
return func(a *AppBuilder[T]) {
a.storeOptions = opts
}
}
39 changes: 32 additions & 7 deletions runtime/v2/module.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ import (
reflectionv1 "cosmossdk.io/api/cosmos/reflection/v1"
appmodulev2 "cosmossdk.io/core/appmodule/v2"
"cosmossdk.io/core/comet"
"cosmossdk.io/core/event"
"cosmossdk.io/core/header"
"cosmossdk.io/core/registry"
"cosmossdk.io/core/server"
Expand All @@ -26,6 +27,7 @@ import (
"cosmossdk.io/log"
"cosmossdk.io/runtime/v2/services"
"cosmossdk.io/server/v2/stf"
rootstore "cosmossdk.io/store/v2/root"
)

var (
Expand All @@ -40,19 +42,19 @@ type appModule[T transaction.Tx] struct {
func (m appModule[T]) IsOnePerModuleType() {}
func (m appModule[T]) IsAppModule() {}

func (m appModule[T]) RegisterServices(registar grpc.ServiceRegistrar) error {
func (m appModule[T]) RegisterServices(registrar grpc.ServiceRegistrar) error {
autoCliQueryService, err := services.NewAutoCLIQueryService(m.app.moduleManager.modules)
if err != nil {
return err
}

autocliv1.RegisterQueryServer(registar, autoCliQueryService)
autocliv1.RegisterQueryServer(registrar, autoCliQueryService)

reflectionSvc, err := services.NewReflectionService()
if err != nil {
return err
}
reflectionv1.RegisterReflectionServiceServer(registar, reflectionSvc)
reflectionv1.RegisterReflectionServiceServer(registrar, reflectionSvc)

return nil
}
Expand Down Expand Up @@ -97,6 +99,7 @@ func init() {
ProvideAppBuilder[transaction.Tx],
ProvideEnvironment[transaction.Tx],
ProvideModuleManager[transaction.Tx],
ProvideStoreBuilder,
),
appconfig.Invoke(SetupAppBuilder),
)
Expand Down Expand Up @@ -146,7 +149,12 @@ type AppInputs struct {
InterfaceRegistrar registry.InterfaceRegistrar
LegacyAmino registry.AminoRegistrar
Logger log.Logger
DynamicConfig server.DynamicConfig `optional:"true"` // can be nil in client wiring
// StoreBuilder is a builder for a store/v2 RootStore satisfying the Store interface
StoreBuilder *StoreBuilder
// StoreOptions are required as input for the StoreBuilder. If not provided, the default options are used.
StoreOptions *rootstore.Options `optional:"true"`
// DynamicConfig can be nil in client wiring, but is required in server wiring.
DynamicConfig server.DynamicConfig `optional:"true"`
}

func SetupAppBuilder(inputs AppInputs) {
Expand All @@ -157,8 +165,22 @@ func SetupAppBuilder(inputs AppInputs) {
app.moduleManager.RegisterInterfaces(inputs.InterfaceRegistrar)
app.moduleManager.RegisterLegacyAminoCodec(inputs.LegacyAmino)

if inputs.DynamicConfig != nil {
inputs.AppBuilder.config = inputs.DynamicConfig
if inputs.DynamicConfig == nil {
return
}
storeOptions := rootstore.DefaultStoreOptions()
if inputs.StoreOptions != nil {
storeOptions = *inputs.StoreOptions
}
var err error
app.db, err = inputs.StoreBuilder.Build(
inputs.Logger,
app.storeKeys,
inputs.DynamicConfig,
storeOptions,
)
if err != nil {
panic(err)
kocubinski marked this conversation as resolved.
Show resolved Hide resolved
}
}

Expand All @@ -178,6 +200,7 @@ func ProvideEnvironment[T transaction.Tx](
appBuilder *AppBuilder[T],
kvFactory store.KVStoreServiceFactory,
headerService header.Service,
eventService event.Service,
) (
appmodulev2.Environment,
store.KVStoreService,
Expand Down Expand Up @@ -209,7 +232,7 @@ func ProvideEnvironment[T transaction.Tx](
env := appmodulev2.Environment{
Logger: logger,
BranchService: stf.BranchService{},
EventService: stf.NewEventService(),
EventService: eventService,
GasService: stf.NewGasMeterService(),
HeaderService: headerService,
QueryRouterService: stf.NewQueryRouterService(),
Expand Down Expand Up @@ -254,10 +277,12 @@ func DefaultServiceBindings() depinject.Config {
}
headerService header.Service = services.NewGenesisHeaderService(stf.HeaderService{})
cometService comet.Service = &services.ContextAwareCometInfoService{}
eventService = stf.NewEventService()
)
return depinject.Supply(
kvServiceFactory,
headerService,
cometService,
eventService,
)
}
56 changes: 56 additions & 0 deletions runtime/v2/store.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,16 @@ package runtime
import (
"errors"
"fmt"
"path/filepath"

"cosmossdk.io/core/server"
"cosmossdk.io/core/store"
"cosmossdk.io/log"
"cosmossdk.io/server/v2/stf"
storev2 "cosmossdk.io/store/v2"
"cosmossdk.io/store/v2/db"
"cosmossdk.io/store/v2/proof"
"cosmossdk.io/store/v2/root"
)

// NewKVStoreService creates a new KVStoreService.
Expand Down Expand Up @@ -59,6 +64,57 @@ type Store interface {
LastCommitID() (proof.CommitID, error)
}

// StoreBuilder is a builder for a store/v2 RootStore satisfying the Store interface.
type StoreBuilder struct {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's smart, I like the abstraction! Makes it more readble too.

store Store
}

// Build creates a new store/v2 RootStore.
func (sb *StoreBuilder) Build(
logger log.Logger,
storeKeys []string,
config server.DynamicConfig,
options root.Options,
) (Store, error) {
if sb.store != nil {
return sb.store, nil
}
home := config.GetString(flagHome)
scRawDb, err := db.NewDB(
db.DBType(config.GetString("store.app-db-backend")),
"application",
filepath.Join(home, "data"),
nil,
)
if err != nil {
return nil, fmt.Errorf("failed to create SCRawDB: %w", err)
}

factoryOptions := &root.FactoryOptions{
Logger: logger,
RootDir: home,
Options: options,
StoreKeys: append(storeKeys, "stf"),
SCRawDB: scRawDb,
}

rs, err := root.CreateRootStore(factoryOptions)
if err != nil {
return nil, fmt.Errorf("failed to create root store: %w", err)
}
sb.store = rs
return sb.store, nil
}

// Get returns the Store. Build must be called before calling Get or the result will be nil.
func (sb *StoreBuilder) Get() Store {
return sb.store
}
kocubinski marked this conversation as resolved.
Show resolved Hide resolved

func ProvideStoreBuilder() *StoreBuilder {
return &StoreBuilder{}
}
kocubinski marked this conversation as resolved.
Show resolved Hide resolved

// StoreLoader allows for custom loading of the store, this is useful when upgrading the store from a previous version
type StoreLoader func(store Store) error

Expand Down
2 changes: 1 addition & 1 deletion runtime/v2/types.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (

const (
ModuleName = "runtime"
FlagHome = "home"
flagHome = "home"
kocubinski marked this conversation as resolved.
Show resolved Hide resolved
)

// validateProtoAnnotations validates that the proto annotations are correct.
Expand Down
19 changes: 10 additions & 9 deletions server/v2/appmanager/appmanager.go
Original file line number Diff line number Diff line change
Expand Up @@ -136,17 +136,16 @@ func (a AppManager[T]) Simulate(ctx context.Context, tx T) (server.TxResult, cor
// Query queries the application at the provided version.
// CONTRACT: Version must always be provided, if 0, get latest
func (a AppManager[T]) Query(ctx context.Context, version uint64, request transaction.Msg) (transaction.Msg, error) {
var (
queryState corestore.ReaderMap
err error
)
// if version is provided attempt to do a height query.
if version != 0 {
queryState, err := a.db.StateAt(version)
if err != nil {
return nil, err
}
return a.stf.Query(ctx, queryState, a.config.QueryGasLimit, request)
queryState, err = a.db.StateAt(version)
} else { // otherwise rely on latest available state.
_, queryState, err = a.db.StateLatest()
}

// otherwise rely on latest available state.
_, queryState, err := a.db.StateLatest()
if err != nil {
return nil, err
}
Expand All @@ -156,6 +155,8 @@ func (a AppManager[T]) Query(ctx context.Context, version uint64, request transa
// QueryWithState executes a query with the provided state. This allows to process a query
// independently of the db state. For example, it can be used to process a query with temporary
// and uncommitted state
func (a AppManager[T]) QueryWithState(ctx context.Context, state corestore.ReaderMap, request transaction.Msg) (transaction.Msg, error) {
func (a AppManager[T]) QueryWithState(
ctx context.Context, state corestore.ReaderMap, request transaction.Msg,
) (transaction.Msg, error) {
return a.stf.Query(ctx, state, a.config.QueryGasLimit, request)
}
2 changes: 0 additions & 2 deletions server/v2/stf/core_header_service.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,6 @@ const headerInfoPrefix = 0x37

// setHeaderInfo sets the header info in the state to be used by queries in the future.
func (s STF[T]) setHeaderInfo(state store.WriterMap, headerInfo header.Info) error {
// TODO storing header info is too low level here, stf should be stateless.
// We should have a keeper that does this.
runtimeStore, err := state.GetWriter(Identity)
if err != nil {
return err
Expand Down
1 change: 1 addition & 0 deletions simapp/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -243,6 +243,7 @@ replace (
cosmossdk.io/api => ../api
cosmossdk.io/client/v2 => ../client/v2
cosmossdk.io/collections => ../collections
cosmossdk.io/core => ../core
cosmossdk.io/store => ../store
cosmossdk.io/tools/confix => ../tools/confix
cosmossdk.io/x/accounts => ../x/accounts
Expand Down
2 changes: 0 additions & 2 deletions simapp/go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,6 @@ cloud.google.com/go/webrisk v1.4.0/go.mod h1:Hn8X6Zr+ziE2aNd8SliSDWpEnSS1u4R9+xX
cloud.google.com/go/webrisk v1.5.0/go.mod h1:iPG6fr52Tv7sGk0H6qUFzmL3HHZev1htXuWDEEsqMTg=
cloud.google.com/go/workflows v1.6.0/go.mod h1:6t9F5h/unJz41YqfBmqSASJSXccBLtD1Vwf+KmJENM0=
cloud.google.com/go/workflows v1.7.0/go.mod h1:JhSrZuVZWuiDfKEFxU0/F1PQjmpnpcoISEXH2bcHC3M=
cosmossdk.io/core v1.0.0-alpha.3 h1:pnxaYAas7llXgVz1lM7X6De74nWrhNKnB3yMKe4OUUA=
cosmossdk.io/core v1.0.0-alpha.3/go.mod h1:3u9cWq1FAVtiiCrDPpo4LhR+9V6k/ycSG4/Y/tREWCY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29 h1:NxxUo0GMJUbIuVg0R70e3cbn9eFTEuMr7ev1AFvypdY=
cosmossdk.io/core/testing v0.0.0-20240923163230-04da382a9f29/go.mod h1:8s2tPeJtSiQuoyPmr2Ag7meikonISO4Fv4MoO8+ORrs=
cosmossdk.io/depinject v1.0.0 h1:dQaTu6+O6askNXO06+jyeUAnF2/ssKwrrszP9t5q050=
Expand Down
Loading
Loading