Enable out-of-order transaction processing for asset table
#83
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
linuskendall
force-pushed
the
triton_partial_updates
branch
from
5438762
to
fcd9054
Compare
|
utACK |
linuskendall
force-pushed
the
triton_partial_updates
branch
from
9726a3e
to
6844d20
Compare
* Update owner and delegate in asset table when collection or creator verification occurs.
* Modify program transformers to enable out-of-order transaction processing by adding the following changes:
* Upsert `leaf` info based on `seq` OR `was_decompressed` flag.
* Upsert `owner` and `delegate` based on `owner_delegate_seq`.
* Upsert "compression status" fields (`compressed`, `compressible`, `supply`, and `supply_mint`) based on `was_decompressed` flag.
* Add `owner_delegate_seq` and `was_decompressed` columns to asset table, and `PROGRAMMABLE_NFT` to `specification_asset_class` type.
* Regenerate Sea ORM objects.
* Needed to temporarily comment out a performance improvement migration that removed foreign keys so that SeaORM CLI would regenerate, since it builds relations based on those constraints.
* Needed to add a migration to add `PROGRAMMABLE_NFT` to `specification_asset_class` since it is needed by the API code but missing from the SQL code.
* Fix docker preparation script to build solana-program-library.
* Also remove unused 'filling' variable.
* Rustfmt/clippy fixes as I change files.
1. Made sure it builds and runs in Docker.
2. Used transaction forwarder to send out of order transactions and then observe asset table in database.
1. Ran a `CreateTree`/`Mint`, `Transfer`, and `Burn` and observed asset table. Ran it in reverse (`Burn`, `Transfer`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order. Compared final state and they match. While running in reverse order I observed that could not get info for the asset via `getAsset` until `Mint` was indexed.
2. Ran a `CreateTree`/`Mint`, `Redeem`, `CancelRedeem`, second `Redeem`, and `Decompress` and observed asset table. Ran it in reverse (`Decompress`, second `Redeem`, `CancelRedeem`, first `Redeem`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order. Compared final state and they match.
3. Ran a `CreateTree`/`Mint`, `Transfer`, and second `Transfer`, and observed asset table. Ran it in reverse (second `Transfer`, first `Transfer`, and then `CreateTree`/`Mint`) and observed database update properly in reverse order. Compared final state and they match.
3. Ran these same transactions in correct forward order on main branch and observed for each asset, final state of asset matched final state of asset when using my PR branch. Observed that running transactions in backwards order on main branch results in various incorrect data in asset table.
4. Ran these same transactions in correct forward order on main branch and observed state of cl_items table was the same as when using my PR branch. Observed that running transactions in backwards order on main branch results in various incorrect data in cl_items table.
linuskendall
force-pushed
the
triton_partial_updates
branch
from
6844d20
to
85ddf35
Compare
linuskendall
pushed a commit
that referenced
this pull request
Jul 26, 2023
* feat(das): add task queue depth metric * count pending tasks only Co-authored-by: Nikhil Acharya <[email protected]> --------- Co-authored-by: Nikhil Acharya <[email protected]>
linuskendall
pushed a commit
that referenced
this pull request
Jul 26, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See: metaplex-foundation#77
Notes
leafinfo based onseqORwas_decompressedflag.owneranddelegatebased onowner_delegate_seq.compressed,compressible,supply, andsupply_mint) based onwas_decompressedflag.Database changes
owner_delegate_seqandwas_decompressedcolumns to asset table, andPROGRAMMABLE_NFTtospecification_asset_classtype.PROGRAMMABLE_NFTtospecification_asset_classsince it is needed by the API code but missing from the SQL code.Other misc. changes
Testing
CreateTree/Mint,Transfer, andBurnand observed asset table. Ran it in reverse (Burn,Transfer, and thenCreateTree/Mint) and observed database update properly in reverse order. Compared final state and they match. While running in reverse order I observed that could not get info for the asset viagetAssetuntilMintwas indexed.CreateTree/Mint,Redeem, andDecompressand observed asset table. Ran it in reverse (Decompress,Redeem, and thenCreateTree/Mint) and observed database update properly in reverse order. Compared final state and they match.CreateTree/Mint,Redeem,CancelRedeem, secondRedeem, andDecompressand observed asset table. Ran it in reverse (Decompress, secondRedeem,CancelRedeem, firstRedeem, and thenCreateTree/Mint) and observed database update properly in reverse order. Compared final state and they match.CreateTree/Mint,Transfer, and secondTransfer, and observed asset table. Ran it in reverse (secondTransfer, firstTransfer, and thenCreateTree/Mint) and observed database update properly in reverse order. Compared final state and they match.