feat(recovery): add crash recovery implementation #3491

Yogesh01000100 · 2024-08-20T21:49:47Z

This PR addresses issue #3114

Changes

Crash Recovery: Updated crash_recovery.proto and related ts files; added core recovery logic (created functions not yet implemented).
Knex Config: Improved knexfile.ts and knexfile-remote.ts; added Docker Compose for production.

RafaelAPB · 2024-08-21T09:04:31Z

I will review this PR

RafaelAPB · 2024-08-23T13:18:40Z

@Yogesh01000100 please rebase with satp-dev (should not have conflicts)

RafaelAPB · 2024-09-02T15:25:35Z

@Yogesh01000100 please include documentation and tests, and update the description, as discussed.

RafaelAPB · 2024-09-16T20:06:25Z

@Yogesh01000100 could you please squash the commits and rebase with latest version of satp-dev, prior to merge?

RafaelAPB

Generally looks very good, but there are some changes to be done prior to merging.
Summarizing my comments:

Add other authors to the commit
Incorporate feedback from the logging process (namely un-hardcoding logs and adding more information)
Implement RollbackState (for example, should state how many more steps are to be rolled-back, at any moment; what was rolledback already; estimated time to completion, etc)
Please add tests that support the new feature
Please add comprehensive documentation on this feature. Example: The readme of SATP should have a section on how to run the docker compose with several examples of configurations.

packages/cactus-plugin-satp-hermes/docker-compose.yaml

packages/cactus-plugin-satp-hermes/package.json

packages/cactus-plugin-satp-hermes/src/main/typescript/core/recovery/crash-manager.ts

packages/cactus-plugin-satp-hermes/src/main/typescript/core/recovery/crash-recovery-handler.ts

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage0-rollback-strategy.ts

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage1-rollback-strategy.ts

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage2-rollback-strategy.ts

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage3-rollback-strategy.ts

...us-plugin-satp-hermes/src/main/typescript/generated/proto/cacti/satp/v02/common/health_pb.ts

@AndreAugusto11

Signed-off-by: Rafael Belchior <[email protected]> feat(bungee): add skeleton for bungee Signed-off-by: André Augusto <[email protected]> refactor(plugin-satp-hermes): update messages formats SATP core version 2.0.0 OpenAPI * created new messages in openAPI definition feat(bungee): fix bungee dependencies Signed-off-by: André Augusto <[email protected]> feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> test(SATP-Hermes): enable decorators needed for gateway coordinator checks Signed-off-by: Rafael Belchior <[email protected]> docs(SATP-Hermes): update package.json Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): remove kotlin sdk, add protobuffer Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): refactor naming Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure improvements Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): update yarn.lock and clean up Signed-off-by: André Augusto <[email protected]> refactor(test-tooling): fix types of streams: use NodeJS.ReadableStream 1. The container management library that we use in the test infrastructure (called dockerode) is expecting streams that are defined in the global namespace of the `@types/node` library, e.g. the standard library of NodeJS itself. 2. Previously we were using the "streams" package to provide type information to the streams that we were passing around to dockerode and it was working fine, but after some changes that seem unrelated this has broken the compilation process. 3. The mentioned changes are not yet on the main branch, but we expect them to be there soon and so this change is laying the groundwork for that by pre-emptively fixing the broken build's root cause which is that the test-tooling package does not declare it's typings related dependencies correctly: It implicitly uses the NodeJS standard library's types but so far had not declared them on the package level. 4. This change is therefore to rectify the issue of the `@types/node` dependency missing from the test-tooling package and also the refactoring of some of the test ledger classes which were relying on the `streams` builtin package instead of correctly using the NodeJS.ReadableStream global. 5. Earlier the reasoning for this was that we try to avoid pulling in types from the global scope because we try to avoid any sort of dependency on the global scope in general. Once we have proof though that this is causing issues with the build, then we must give up the principle for practical reasons (and only in the minimum viable scope, e.g. this does not change the fact that everywhere else in the codebase we should still do our best to avoid using the global scoped classes, types, functions, etc..). Thank you to @AndreAugusto11 and @RafaelAPB for pointing out this issue through the pull request of his that is currently being worked on at the time of this writing: RafaelAPB#72 Related to but does not address hyperledger-cacti#2811 Signed-off-by: Peter Somogyvari <[email protected]> refactor(SATP-Hermes): remove unused packages * updated openapi version * updated bungee plugin version to 2.0.0-alpha.2 Co-authored-by: Peter Somogyvari <[email protected]> Signed-off-by: André Augusto <[email protected]> Signed-off-by: Peter Somogyvari <[email protected]>

Co-authored-by: André Augusto <[email protected]> Co-authored-by: Peter Somogyvari <[email protected]> Co-authored-by: Rafael Belchior <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> test(satp-hermes): fix hanging tests Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): fix cspell and lint Signed-off-by: André Augusto <[email protected]>

@AndreAugusto11

Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): BLO API specification Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): add BLO layer Signed-off-by: Rafael Belchior <[email protected]> fix(satp-hermes): fix status endpoint and spec Signed-off-by: Rafael Belchior <[email protected]> fix(satp-hermes): fix lint Signed-off-by: André Augusto <[email protected]> ci(satp-hermes): rebase and update ci Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): update BLO specification Signed-off-by: Rafael Belchior <[email protected]> docs(satp-hermes): migrate BLO open api spec from json to yml Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): update Go BLO SDK generation Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): update BLO specification Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): add health check endpoint to BLO spec Signed-off-by: Rafael Belchior <[email protected]> fix(satp-hermes): re-add open-api json files The canonical source for the API definition are the yaml files located at packages/cactus-plugin-satp-hermes/src/main/yml/openapi.yml, due to its better readability compared to json. When yarn codegen is ran within the cactus-plugin-satp-hermes package, the json files are generated from the yaml. Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): remove deprecated spec Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): add gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> feat(bungee): add skeleton for bungee Signed-off-by: André Augusto <[email protected]> refactor(plugin-satp-hermes): update messages formats SATP core version 2.0.0 OpenAPI * created new messages in openAPI definition feat(bungee): fix bungee dependencies Signed-off-by: André Augusto <[email protected]> feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> test(SATP-Hermes): enable decorators needed for gateway coordinator checks Signed-off-by: Rafael Belchior <[email protected]> docs(SATP-Hermes): update package.json Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): remove kotlin sdk, add protobuffer Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): refactor naming Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure improvements Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): update yarn.lock and clean up Signed-off-by: André Augusto <[email protected]> refactor(test-tooling): fix types of streams: use NodeJS.ReadableStream 1. The container management library that we use in the test infrastructure (called dockerode) is expecting streams that are defined in the global namespace of the `@types/node` library, e.g. the standard library of NodeJS itself. 2. Previously we were using the "streams" package to provide type information to the streams that we were passing around to dockerode and it was working fine, but after some changes that seem unrelated this has broken the compilation process. 3. The mentioned changes are not yet on the main branch, but we expect them to be there soon and so this change is laying the groundwork for that by pre-emptively fixing the broken build's root cause which is that the test-tooling package does not declare it's typings related dependencies correctly: It implicitly uses the NodeJS standard library's types but so far had not declared them on the package level. 4. This change is therefore to rectify the issue of the `@types/node` dependency missing from the test-tooling package and also the refactoring of some of the test ledger classes which were relying on the `streams` builtin package instead of correctly using the NodeJS.ReadableStream global. 5. Earlier the reasoning for this was that we try to avoid pulling in types from the global scope because we try to avoid any sort of dependency on the global scope in general. Once we have proof though that this is causing issues with the build, then we must give up the principle for practical reasons (and only in the minimum viable scope, e.g. this does not change the fact that everywhere else in the codebase we should still do our best to avoid using the global scoped classes, types, functions, etc..). Thank you to @AndreAugusto11 and @RafaelAPB for pointing out this issue through the pull request of his that is currently being worked on at the time of this writing: RafaelAPB#72 Related to but does not address hyperledger-cacti#2811 Signed-off-by: Peter Somogyvari <[email protected]> refactor(SATP-Hermes): remove unused packages * updated openapi version * updated bungee plugin version to 2.0.0-alpha.2 Co-authored-by: Peter Somogyvari <[email protected]> Signed-off-by: André Augusto <[email protected]> Signed-off-by: Peter Somogyvari <[email protected]> fix(satp-hermes): re-add open-api json files The canonical source for the API definition are the yaml files located at packages/cactus-plugin-satp-hermes/src/main/yml/openapi.yml, due to its better readability compared to json. When yarn codegen is ran within the cactus-plugin-satp-hermes package, the json files are generated from the yaml. Signed-off-by: Rafael Belchior <[email protected]> ci(satp-hermes): re-activate satp-hermes ci Signed-off-by: Rafael Belchior <[email protected]> fixup: update package json, ci, yarn lock Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): add gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> feat(bungee): add skeleton for bungee Signed-off-by: André Augusto <[email protected]> refactor(plugin-satp-hermes): update messages formats SATP core version 2.0.0 OpenAPI * created new messages in openAPI definition feat(bungee): fix bungee dependencies Signed-off-by: André Augusto <[email protected]> feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator feat(SATP-Hermes): gateway coordinator WIP Signed-off-by: Rafael Belchior <[email protected]> test(SATP-Hermes): enable decorators needed for gateway coordinator checks Signed-off-by: Rafael Belchior <[email protected]> docs(SATP-Hermes): update package.json Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): re-estructure project Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): remove kotlin sdk, add protobuffer Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure Signed-off-by: Rafael Belchior <[email protected]> refactor(SATP-Hermes): refactor naming Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): protobuf structure improvements Signed-off-by: Rafael Belchior <[email protected]> feat(SATP-Hermes): update yarn.lock and clean up Signed-off-by: André Augusto <[email protected]> refactor(test-tooling): fix types of streams: use NodeJS.ReadableStream 1. The container management library that we use in the test infrastructure (called dockerode) is expecting streams that are defined in the global namespace of the `@types/node` library, e.g. the standard library of NodeJS itself. 2. Previously we were using the "streams" package to provide type information to the streams that we were passing around to dockerode and it was working fine, but after some changes that seem unrelated this has broken the compilation process. 3. The mentioned changes are not yet on the main branch, but we expect them to be there soon and so this change is laying the groundwork for that by pre-emptively fixing the broken build's root cause which is that the test-tooling package does not declare it's typings related dependencies correctly: It implicitly uses the NodeJS standard library's types but so far had not declared them on the package level. 4. This change is therefore to rectify the issue of the `@types/node` dependency missing from the test-tooling package and also the refactoring of some of the test ledger classes which were relying on the `streams` builtin package instead of correctly using the NodeJS.ReadableStream global. 5. Earlier the reasoning for this was that we try to avoid pulling in types from the global scope because we try to avoid any sort of dependency on the global scope in general. Once we have proof though that this is causing issues with the build, then we must give up the principle for practical reasons (and only in the minimum viable scope, e.g. this does not change the fact that everywhere else in the codebase we should still do our best to avoid using the global scoped classes, types, functions, etc..). Thank you to @AndreAugusto11 and @RafaelAPB for pointing out this issue through the pull request of his that is currently being worked on at the time of this writing: RafaelAPB#72 Related to but does not address hyperledger-cacti#2811 Signed-off-by: Peter Somogyvari <[email protected]> refactor(SATP-Hermes): remove unused packages * updated openapi version * updated bungee plugin version to 2.0.0-alpha.2 Co-authored-by: Peter Somogyvari <[email protected]> Signed-off-by: André Augusto <[email protected]> Signed-off-by: Peter Somogyvari <[email protected]>

Authored by: Carlos Amaro <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): satp proto update (hyperledger-cacti#3147) * feat(satp-hermes): update protos Signed-off-by: Carlos Amaro <[email protected]> * feat(satp-hermes): protos Signed-off-by: Carlos Amaro <[email protected]> * feat(satp-hermes): protos Signed-off-by: Carlos Amaro <[email protected]> --------- Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): update protos Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): core stage services message names update Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): gRPC server start implementation and gateway_refactor in handlers Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): satp proto update also updates yarn lock Signed-off-by: Rafael Belchior <[email protected]> fix(satp-hermes): update yarn, comment method fix(satp-hermes): update yarn ci(satp-hermes): fix bungee and satp ci Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]> build(satp-hermes): add gateway client Go SDK Signed-off-by: Rafael Belchior <[email protected]> feat(satp-hermes): re-add message proto Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): stage1-service and handler refactored Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes):stage2-service and handler refactored server-side Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): service handlers server Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): client stage handlers refactor Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

* Views have new attribute viewProof * This attribute has the roots of all state proofs and transaction proofs in the view * Added new endpoint to verify integrity of view proofs * Created strategy for bungee-hermes to support the ethereum-connector * Created new test for the strategy-ethereum, and altered the API test to test the new viewProof endpoint * Updated documentation to reflect new changes. * Fixes in Fabric Strategy, stateProofs and error handling Co-authored-by: André Augusto <[email protected]> Co-authored-by: Rafael Belchior <[email protected]> Signed-off-by: eduv09 <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Iulia Mihaiu <[email protected]> Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): add methods to fabric connector Signed-off-by: Iulia Mihaiu <[email protected]> fix(prometheus): metrics.ts leaks to global registry hyperledger-cacti#1202 1. Specified a `register` property of the gauges as an empty array so that it does not pollute the global namespace. This is how this is supposed to be done as per the docs of prom-client. 2. Once the change from 1) took place, the issue became that the metrics gathering code was still trying to hit up the global scope for the metrics, e.g. calling the get metrics methods directly on the promClient object instead of the registry that we create for each prmoetheus exporter object separately. So a little additional refactor ensued to fix this as well by making sure that we grab a reference of the registry object at construction time and then re-use that wherever needed instead of going through the global promClient object. 3. Added missing .startMetricsCollection calls in the plugin constructors to ensure that the prometheus exporter object gets initialized properly (this is where the registry gets created as well so without this there are crashes happening when one tries to access the metrics through the registry) Why though? The problem was that the metrics.ts file that we have for all the plugin's metrics constructs a new Metric (Gauge) object at import time which then defaults to registering the metric in the global registry of prom-client by default. The latter was causing crashes when separate versions of the same metrics.ts file are imported in a scenario were the API server imports plugins from a different directory (this issue is coming from the branch where I'm working on plugin sandboxing via the live-plugin-manager). Fixes hyperledger-cacti#1202 Signed-off-by: Peter Somogyvari <[email protected]> feat(cc-tx-viz): added test for multiple prometheus metrics on fabric Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): add metric model Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): added croschainEventLog models Signed-off-by: Iulia Mihaiu <[email protected]> feat(fabric): added endpoint for list of transaction receipts Signed-off-by: Iulia Mihaiu <[email protected]> feat(test-tooling): add RabbitMQ test server feat(cc-tx-viz): add RabbitMQ support; update model feat(cc-tx-viz): add preliminary support to Fabric and Besu build(deps): upgrade axios to latest to fix CVE Details CVE-2021-3749 high severity Vulnerable versions: <= 0.21.1 Patched version: 0.21.2 axios is vulnerable to Inefficient Regular Expression Complexity Signed-off-by: Peter Somogyvari <[email protected]> feat(cc-tx-viz): add methods to fabric connector Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): add metric model Signed-off-by: Iulia Mihaiu <[email protected]> xfix(cc-tx-viz): error implementation of IsVisualizable feat(cc-tx-viz): add methods to fabric and besu; update models Signed-off-by: Iulia Mihaiu <[email protected]> feat(cc-tx-viz): update cross chain events log for fabric transactions Signed-off-by: Iulia Mihaiu <[email protected]> feat(cctxviz): fix fabric prometheus feat(cctxviz): fix besu prometheus feat(cctxviz): fix fabric test feat(cctxviz): update tx receipt feat(cctxviz): add test script Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add test receipts feat(cctxviz): refactor CrossChainEventLog Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): refactor tests feat(cc-tx-viz): update model, add tests chore(cc-tx-viz): update dependencies Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): persist logs Signed-off-by: Rafael Belchior <[email protected]> test(cc-tx-viz): update test Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): update strinfify to csv Signed-off-by: Rafael Belchior <[email protected]> chore(cc-tx-viz): add csv folder Signed-off-by: Rafael Belchior <[email protected]> chore(cc-tx-viz): move csv folder Signed-off-by: Rafael Belchior <[email protected]> feat(cc-tx-viz): add test feat(cctxviz): add csv file, process mining script Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): proof of concept v1 Signed-off-by: Rafael Belchior <[email protected]> chore(cctxviz): update csv Signed-off-by: Rafael Belchior <[email protected]> docs(cctxviz): update readme Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update tests Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add carbon footprint Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update tests Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update model Signed-off-by: Rafael Belchior <[email protected]> feat(fabric-connector): update default queue Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): fabric test Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add parameters to fabric calls Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): support besu Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update metrics Signed-off-by: Rafael Belchior <[email protected]> test(cctxvi): update fabric and besu tests Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): besu and fabric Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): support call transactions on fabric Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add aggregation basis Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): aggregate cctx Signed-off-by: Rafael Belchior <[email protected]> chore: update yarn and remove package.lock Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): containerize draft Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add revenue Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update model playbook Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add dummy use case playbook Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update persist test and dummy uc Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add timer Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): workload test Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add more timers Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): add timer to map on connectors Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update testing framework Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): vis plot wip Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): evaluate baseline Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update plots Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add test files Signed-off-by: Rafael Belchior <[email protected]> feat(cctxviz): update testing framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): time-efficient polling receipts Signed-off-by: Rafael Belchior <[email protected]> chore(cctxviz): add test chaincode Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update test Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): use case v1 Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update test framework Signed-off-by: Rafael Belchior <[email protected]> build(cc-tx-visualization): added missing fabric-contract-api This will fix one compilation error. Signed-off-by: Peter Somogyvari <[email protected]> chore(deps): updated yarn.lock after rebase onto upstream/main Signed-off-by: Peter Somogyvari <[email protected]> build(cc-tx-visualization): deleted redundant test fixtures Also added the test-tooling package as a dev dependency which should fix a few of the compiler errors we were getting on account of missing dependencies. Signed-off-by: Peter Somogyvari <[email protected]> fix: resolved build errors for cctxviz branch - test-tooling package changed that needs to be reverted before commmit to main branch - added import statement for test data generationt xfix(cc-tx-viz): error implementation of IsVisualizable chore: update yarn and remove package.lock Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): testing framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): test framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): update framework Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add graphs Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add test Signed-off-by: Rafael Belchior <[email protected]> fix(cctxviz): fix bug Signed-off-by: Rafael Belchior <[email protected]> test(cctxviz): add process conformance Signed-off-by: Rafael Belchior <[email protected]>

The plugin now utilizes RxJS instead of RabbitMQ in transaction monitoring. ReplaySubjects store and emit observed transactional data to subscribers. fix(cc-tx-visualization): rebase latest version Signed-off-by: Rafael Belchior <[email protected]> squash! - migrate a test case to Fabric v2.5.6 LTS AIO This is just a snippet of a change from the pair programming call with Bruno, it can be safely squashed, no worries. Signed-off-by: Peter Somogyvari <[email protected]> Signed-off-by: Bruno Mateus <[email protected]> feat(cc-tx-visualization): ccmodel serialization with PM4PY The plugin now creates the ccmodel for conformance checking with PM4PY. Signed-off-by: Bruno Mateus <[email protected]>

The plugin is now able to: Create a Cross-Chain Model based on monitored transactions Compare new transactions against the model and detect non-conformance Signed-off-by: Bruno Mateus <[email protected]>

Signed-off-by: eduv09 <[email protected]>

bug correction session functionalities Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): errors file and other fixes Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): fix packages versions Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Rafael Belchior <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): satp rejection Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): handler bug correction and errors Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): bridge erros and log messages Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): some error fixes Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): some fixes requested Signed-off-by: Carlos Amaro <[email protected]> Signed-off-by: Rafael Belchior <[email protected]>

Co-authored-by: eduv09 <[email protected]> Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): error messages in stage0 Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): service tests added stage0 and error handling Signed-off-by: Carlos Amaro <[email protected]> feat(satp-hermes): test e2e Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): fix bugs and test e2e working transfer Signed-off-by: Carlos Amaro <[email protected]> fix(stap-hermes): bugs and 2 gateway test Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]>

Signed-off-by: Eduardo Vasques <[email protected]>

Signed-off-by: Carlos Amaro <[email protected]> refactor(cbdc-example): update with new satp-plugin Signed-off-by: eduv09 <[email protected]> refactor(cbdc-example): fixes Signed-off-by: Carlos Amaro <[email protected]> <

Signed-off-by: Eduardo Vasques <[email protected]> fix(satp-hermes): bug fixing Signed-off-by: Carlos Amaro <[email protected]> fix(cbdc-example): bug fixing Signed-off-by: Carlos Amaro <[email protected]> feat(cbdc): final fixes Signed-off-by: Eduardo Vasques <[email protected]> fix(cbdc-example): remove unecessary variable Signed-off-by: Carlos Amaro <[email protected]> fix(satp-hermes): remove unessary logs Signed-off-by: Carlos Amaro <[email protected]> fix(cbdc-example): docker ports Signed-off-by: Carlos Amaro <[email protected]>

* Separated Transfer functionality into local blockchain transfers and cross-chain transfers * Added new table with the state of token approvals made to the bridge * Highlight cross-chain functionality with red coloring * add helper page with instructions on how to use app Signed-off-by: André Augusto <[email protected]>

* Created new Docker Image and updated README.md Signed-off-by: André Augusto <[email protected]>

Also fixing a few other bugs which were all necessary to make it work after a rebase onto upstream/main as of 2024-10-10. 1. The tsconfig.json file in the root did not contain a reference to the package at `packages/cactus-plugin-ccmodel-hephaestus` so it was being excluded from the compilation. 2. express version was not up to date with the rest of the monorepo which Signed-off-by: Peter Somogyvari <[email protected]>

Import paths were invalid possibly because the files were moved around within the directory tree and the imports never got updated. There are still imports which are invalid because of issues with the remix IDE test library being used but not declared as a dependency. I've opened a separate issue about fixing those problems here: hyperledger-cacti#3585 Signed-off-by: Peter Somogyvari <[email protected]>

…ervices, ui Signed-off-by: Carlos Amaro <[email protected]>

* added logic for satp-bridge supporting the ethereum connector * finished the integration with bungee * changed common stringify for safeStableStringify across the package * added test for the new bridge & end-to-end tests for the new features * fabric-connector: fix bug handling empty read-write sets * bungee: strategy fabric fix bug handling empty read-write sets * bungee: hole package introduce safeStableStringify * ethereum-connector: bump web3 packages versions to fix (web3/web3.js#6798) Signed-off-by: Eduardo Vasques <[email protected]>

Signed-off-by: Yogesh01000100 <[email protected]> chore(satp-hermes): improve DB management Signed-off-by: Rafael Belchior <[email protected]> chore(satp-hermes): crash recovery architecture Signed-off-by: Rafael Belchior <[email protected]> fix(recovery): enhance crash recovery and rollback implementation Signed-off-by: Yogesh01000100 <[email protected]> refactor(recovery): consolidate logic and improve SATP message handling Signed-off-by: Yogesh01000100 <[email protected]> feat(recovery): add rollback implementations Signed-off-by: Yogesh01000100 <[email protected]> fix: correct return types and inits Signed-off-by: Yogesh01000100 <[email protected]> Co-authored-by: Rafael Belchior <[email protected]>

Signed-off-by: Yogesh01000100 <[email protected]>

LordKubaya

Review how sessionData is being used, and take a look at the Stage 3 question.
Please document the new code as well. The rest is being documented in this PR:
https://github.com/hyperledger/cacti/pull/3619

LordKubaya · 2024-11-06T18:29:58Z

packages/cactus-plugin-satp-hermes/src/main/typescript/core/recovery/crash-manager.ts

+
+  private async checkCrash(session: SATPSession): Promise<CrashStatus> {
+    const fnTag = `${this.className}#checkCrash()`;
+    const sessionData = session.hasClientSessionData()


Why does this prioritize client session?
In this Implementation the gateway can be a client and a server at the same time. So, when this is the case we may not be deteting some crashes.

LordKubaya · 2024-11-06T18:30:20Z

packages/cactus-plugin-satp-hermes/src/main/typescript/core/recovery/crash-manager.ts

+  public async checkAndResolveCrash(session: SATPSession): Promise<void> {
+    const fnTag = `${this.className}#checkAndResolveCrash()`;
+
+    const sessionData = session.hasClientSessionData()


LordKubaya · 2024-11-06T18:35:19Z

packages/cactus-plugin-satp-hermes/src/main/typescript/core/recovery/crash-manager.ts

+  public async handleRecovery(session: SATPSession): Promise<boolean> {
+    const fnTag = `${this.className}#handleRecovery()`;
+
+    const sessionData = session.hasClientSessionData()


LordKubaya · 2024-11-06T20:36:13Z

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage0-rollback-strategy.ts

+      throw new Error(`${fnTag}, session data is not correctly initialized`);
+    }
+    const sessionData = session.hasClientSessionData()
+      ? session.getClientSessionData()


LordKubaya · 2024-11-06T20:44:21Z

...us-plugin-satp-hermes/src/main/typescript/core/recovery/rollback/stage3-rollback-strategy.ts

+
+      this.log.info(`${fnTag} Asset Id: ${assetId} amount: ${amount}`);
+
+      await bridgeManager.burnAsset(assetId, Number(amount));


In the Stage 3 rollback, the rollback is only feasible if it occurs before the asset is minted on the receiver chain. Once minting happens, if the gateway encounters an issue and fails to assign the minted amount to the recipient, a rollback can no longer be initiated. Is this the reason why a rollback isn’t considered in such cases?

Wouldn't it make sense, then, for the minted amount on the receiver chain to be burned and re-minted on the source chain too?

LordKubaya · 2024-11-06T20:47:11Z

packages/cactus-plugin-satp-hermes/src/test/typescript/unit/recovery/logging.test.ts

+    client: true,
+  });
+
+  const sessionData = mockSession.hasClientSessionData()


We need to consider this carefully. If sessionData remains as it is, we must handle it with care and clearly differentiate between the client and server sides of the gateway. I designed the sessionData this way to ensure that a gateway can act as both a client and server to itself.

LordKubaya · 2024-11-06T20:48:04Z

packages/cactus-plugin-satp-hermes/src/test/typescript/unit/recovery/services.test.ts

+    expect(result).toBe(true);
+  });
+
+  /*it("intitiate rollback function test", async () => {


is there a reason for this to be commented out?

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch from f9014b0 to 0de9744 Compare August 21, 2024 20:08

RafaelAPB force-pushed the satp-dev branch from 5744312 to c21957b Compare August 23, 2024 13:16

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch from 0de9744 to 4c0124d Compare August 23, 2024 17:19

Yogesh01000100 changed the title ~~feat: add crash recovery and knex config for production~~ feat(recovery): add crash recovery implementation Aug 25, 2024

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch from ce9a179 to 24b8eaf Compare September 8, 2024 07:44

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch from 24b8eaf to 728e7cb Compare September 16, 2024 18:56

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch 2 times, most recently from 1a55673 to 21ad772 Compare September 17, 2024 10:11

RafaelAPB requested changes Sep 17, 2024

View reviewed changes

RafaelAPB and others added 17 commits October 10, 2024 18:01

fixup(satp-hermes): remove test protos and endpoints

37426e9

Signed-off-by: Rafael Belchior <[email protected]>

fixup(satp-hermes): add skeleton for satp-bridge

b4f191f

Signed-off-by: Rafael Belchior <[email protected]>

chore(satp-hermes): rename generated code for gateway and blo

53ee40f

Signed-off-by: Rafael Belchior <[email protected]>

fix(satp-hermes): fix knex path for old gateway tests

63807c4

Signed-off-by: Rafael Belchior <[email protected]>

feat(satp-hermes): update protos and services

a8dff06

Signed-off-by: Carlos Amaro <[email protected]>

fixup!: wip gateway refactoring

0050181

Signed-off-by: Rafael Belchior <[email protected]>

fixup!: remove old implementation

acfcc16

Signed-off-by: Rafael Belchior <[email protected]>

fixup!: rename new implementation, add new public api for plugin

42512eb

Signed-off-by: Rafael Belchior <[email protected]>

fix: refactor services and handlers

4ec94bb

Signed-off-by: Rafael Belchior <[email protected]>

fix: eslint all

ef2b66d

Signed-off-by: Rafael Belchior <[email protected]>

maramih and others added 19 commits October 10, 2024 19:09

feat(hephaestus): analyse transactions using hephaestus

82b1c23

The plugin is now able to: Create a Cross-Chain Model based on monitored transactions Compare new transactions against the model and detect non-conformance Signed-off-by: Bruno Mateus <[email protected]>

feat(satp-hermes): add skeleton for stage0

eb30bf5

Signed-off-by: eduv09 <[email protected]>

feat(satp-hermes): add satp-plugin cli (wip)

b0d76ac

Signed-off-by: Rafael Belchior <[email protected]>

chore(satp-hermes): fix satp build process

7121d90

Signed-off-by: Rafael Belchior <[email protected]>

feat(satp-hermes): satp-manager and gol refactor

c8e859f

Signed-off-by: Carlos Amaro <[email protected]>

fix(satp-hermes): fix handler and service intialization

053db77

Signed-off-by: Rafael Belchior <[email protected]>

feat(satp-hermes): openApi transact implementation

9b5069e

Signed-off-by: Carlos Amaro <[email protected]>

feat(satp-hermes): improve admin endpoints

7fc40a7

Signed-off-by: Eduardo Vasques <[email protected]>

refactor(cbdc-example): added gateways and contracts

49dbcad

Signed-off-by: Carlos Amaro <[email protected]> refactor(cbdc-example): update with new satp-plugin Signed-off-by: eduv09 <[email protected]> refactor(cbdc-example): fixes Signed-off-by: Carlos Amaro <[email protected]> <

docs(cbdc-example): remove unused variables and unused code

426ede8

* Created new Docker Image and updated README.md Signed-off-by: André Augusto <[email protected]>

petermetz force-pushed the satp-dev branch from 3ad9c6c to 34c81e7 Compare October 11, 2024 19:20

refactor(cbdc-example): use oapi sdk instead of axios, cbdc-backend s…

dd47d6a

…ervices, ui Signed-off-by: Carlos Amaro <[email protected]>

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch 2 times, most recently from 49e1135 to fb703b4 Compare October 16, 2024 19:57

eduv09 and others added 4 commits November 1, 2024 17:28

fix: add unit tests and resolve rollbackstate

cde6b2e

Signed-off-by: Yogesh01000100 <[email protected]>

feat: add function processing logs from g2

b30ccb5

Signed-off-by: Yogesh01000100 <[email protected]>

Yogesh01000100 force-pushed the feature/crash-recovery-improvements branch from fb703b4 to b30ccb5 Compare November 3, 2024 19:16

LordKubaya reviewed Nov 6, 2024

View reviewed changes

RafaelAPB force-pushed the satp-dev branch from 3d863f3 to 13e0302 Compare November 9, 2024 01:30

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(recovery): add crash recovery implementation #3491

feat(recovery): add crash recovery implementation #3491

Yogesh01000100 commented Aug 20, 2024 •

edited

Loading

RafaelAPB commented Aug 21, 2024

RafaelAPB commented Aug 23, 2024

RafaelAPB commented Sep 2, 2024

RafaelAPB commented Sep 16, 2024

RafaelAPB left a comment •

edited

Loading

LordKubaya left a comment

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024

LordKubaya Nov 6, 2024


		this.log.info(`${fnTag} Asset Id: ${assetId} amount: ${amount}`);

		await bridgeManager.burnAsset(assetId, Number(amount));

feat(recovery): add crash recovery implementation #3491

Are you sure you want to change the base?

feat(recovery): add crash recovery implementation #3491

Conversation

Yogesh01000100 commented Aug 20, 2024 • edited Loading

Changes

RafaelAPB commented Aug 21, 2024

RafaelAPB commented Aug 23, 2024

RafaelAPB commented Sep 2, 2024

RafaelAPB commented Sep 16, 2024

RafaelAPB left a comment • edited Loading

Choose a reason for hiding this comment

LordKubaya left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Yogesh01000100 commented Aug 20, 2024 •

edited

Loading

RafaelAPB left a comment •

edited

Loading