Skip to content

Commit

Permalink
Enable out-of-order transaction processing for asset table #77
Browse files Browse the repository at this point in the history
* Update owner and delegate in asset table when collection or creator verification occurs.
* Modify program transformers to enable out-of-order transaction processing by adding the following changes:
  * Upsert `leaf` info based on `seq` OR `was_decompressed` flag.
  * Upsert `owner` and `delegate` based on `owner_delegate_seq`.
  * Upsert "compression status" fields (`compressed`, `compressible`, `supply`, and `supply_mint`) based on `was_decompressed` flag.

* Add `owner_delegate_seq` and `was_decompressed` columns to asset table, and `PROGRAMMABLE_NFT` to `specification_asset_class` type.
  * Regenerate Sea ORM objects.
  * Needed to temporarily comment out a performance improvement migration that removed foreign keys so that SeaORM CLI would regenerate, since it builds relations based on those constraints.
  * Needed to add a migration to add `PROGRAMMABLE_NFT` to `specification_asset_class` since it is needed by the API code but missing from the SQL code.

* Fix docker preparation script to build solana-program-library.
* Also remove unused 'filling' variable.
* Rustfmt/clippy fixes as I change files.

  1. Made sure it builds and runs in Docker.
  2. Used transaction forwarder to send out of order transactions and then observe asset table in database.
     1.  Ran a `CreateTree`/`Mint`, `Transfer`, and `Burn` and observed asset table.  Ran it in reverse (`Burn`, `Transfer`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order.  Compared final state and they match.  While running in reverse order I observed that could not get info for the asset via `getAsset` until `Mint` was indexed.
     2.  Ran a `CreateTree`/`Mint`, `Redeem`, `CancelRedeem`, second `Redeem`, and `Decompress` and observed asset table.  Ran it in reverse (`Decompress`, second `Redeem`, `CancelRedeem`, first `Redeem`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order.  Compared final state and they match.
     3.  Ran a `CreateTree`/`Mint`, `Transfer`, and second `Transfer`, and observed asset table.  Ran it in reverse (second `Transfer`, first `Transfer`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order.  Compared final state and they match.
  3.  Ran these same transactions in correct forward order on main branch and observed for each asset, final state of asset matched final state of asset when using my PR branch.  Observed that running transactions in backwards order on main branch results in various incorrect data in asset table.
  4.  Ran these same transactions in correct forward order on main branch and observed state of cl_items table was the same as when using my PR branch.  Observed that running transactions in backwards order on main branch results in various incorrect data in cl_items table.
  • Loading branch information
linuskendall committed Jul 4, 2023
1 parent 2356da7 commit 6844d20
Show file tree
Hide file tree
Showing 37 changed files with 5,537 additions and 380 deletions.
12 changes: 11 additions & 1 deletion Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

53 changes: 36 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ Then with a local `DATABASE_URL` var exported like this `export DATABASE_URL=pos

If you need to install `sea-orm-cli` run `cargo install sea-orm-cli`.

Note: The current SeaORM types were generated using version 0.9.3 so unless you want to upgrade you can install using `cargo install sea-orm-cli --version 0.9.3`.

Also note: The migration `m20230224_093722_performance_improvements` needs to be commented out of the migration lib.rs in order for the Sea ORM `Relations` to generate correctly.

#### Developing Locally
*Prerequisites*
* A Postgres Server running with the database setup according to ./init.sql
Expand Down Expand Up @@ -153,25 +157,40 @@ And a Metrics System on
http://localhost:3000
```

Here is an example request to the API
Here are some example requests to the Read API:

```bash
curl --request POST \
--url http://localhost:9090 \
--header 'Content-Type: application/json' \
--data '{
"jsonrpc": "2.0",
"method":"get_assets_by_owner",
"id": "rpd-op-123",
"params": [
"CMvMqPNKHikuGi7mrngvQzFeQ4rndDnopx3kc9drne8M",
"created",
50,
1,
"",
""
]
}'
curl --request POST --url http://localhost:9090 --header 'Content-Type: application/json' --data '{
"jsonrpc": "2.0",
"method": "getAssetsByOwner",
"params": [
"CMvMqPNKHikuGi7mrngvQzFeQ4rndDnopx3kc9drne8M",
{ "sortBy": "created", "sortDirection": "asc"},
50,
1,
"",
""
],
"id": 0
}' | json_pp

curl --request POST --url http://localhost:9090 --header 'Content-Type: application/json' --data '{
"jsonrpc": "2.0",
"method": "getAsset",
"params": [
"8vw7tdLGE3FBjaetsJrZAarwsbc8UESsegiLyvWXxs5A"
],
"id": 0
}' | json_p

curl --request POST --url http://localhost:9090 --header 'Content-Type: application/json' --data '{
"jsonrpc": "2.0",
"method": "getAssetProof",
"params": [
"8vw7tdLGE3FBjaetsJrZAarwsbc8UESsegiLyvWXxs5A"
],
"id": 0
}' | json_pp
```

# Deploying to Kubernetes
Expand Down
2 changes: 1 addition & 1 deletion das_api/src/api/api_impl.rs
Original file line number Diff line number Diff line change
Expand Up @@ -278,7 +278,7 @@ impl ApiContract for DasApi {
page,
before,
after,
json_uri
json_uri,
} = payload;
// Deserialize search assets query
self.validate_pagination(&limit, &page, &before, &after)?;
Expand Down
Loading

0 comments on commit 6844d20

Please sign in to comment.