Skip to content

Koios 1.1.0
Compare
Choose a tag to compare
@rdlrt rdlrt released this 06 Dec 23:48
· 35 commits to main since this release
v1.1.0
dd3bdc3
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Learn about vigilant mode.

Koios 1.1.0

You can refer to here if looking for difference between koios-1.1.0 and koios-1.1.0rc, mostly chores behind-the-scenes

Koios v1.1.0 is the first major [breaking] release for Koios consumers in a while, and is rolled out under new base prefix (/api/v1).
The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. To keep migration paths easier, we have added some v0 re-directors to ensure v0 applications do not break, but users are absolutely recommended to upgrade from deprecated v0, which could be retired in future months (likely with next breaking release at Conway).

While we understand there is still some scope for improvement especially on asset query performance (though you should see quite some improvements already, alongwith easier filtering on *_utxos), that will be one of our primary focus points on upcoming Koios Lite project.

For instance providers, unless coming from v1.1.0rc, the underlying infrastructure would've changed drastically - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network Date URL Specs Release gRest Instance Release DBSync Node
GuildNet 24-11-2023 22:00 UTC https://guild.koios.rest/ v1.1.0 koios-1.1.0 13.1.1.3 8.1.2
Preview 28-11-2023 22:00 UTC https://preview.koios.rest/ v1.1.0 koios-1.1.0 13.1.1.3 8.1.2
PreProd 08-12-2023 22:00 UTC https://preprod.koios.rest/ v1.1.0 koios-1.1.0 13.1.1.3 8.1.2
Mainnet 13-12-2023 22:00 UTC https://api.koios.rest/ v1.1.0 koios-1.1.0 13.1.1.3 8.1.2

Changes for API (from v1.0.11)

Expand

Deprecations

  • /tx_utxos - No longer required as replaced by /utxo_info #227

New endpoints added

  • /pool_registrations - List of all pool registrations initiated in the requested epoch #239
  • /pool_retirements - List of all pool retirements initiated in the requested epoch #239
  • /treasury_withdrawals - List of withdrawals made from treasury #239
  • /reserve_withdrawals - List of withdrawals made from reserves (MIRs) #239
  • /account_txs - Transactions associated with a given stake address #239
  • /address_utxos - Get UTxO details for requested addresses #239
  • /asset_utxos - Get UTxO details for requested assets #239
  • /script_utxos - Get UTxO details for requested script hashes #239
  • /utxo_info - Details for requested UTxO arrays #239
  • /script_info - Information about a given script FROM script hashes #239
  • /ogmios - Expose Ogmios endpoints ( cardano-community/guild-operators#1690 )

Data Input/Output Changes

  • Input - /account_utxos , /credential_utxos - Accept extended as an additional flag - which enables asset_list, reference_script and inline_datum to the output #239
  • Output - /block_txs - Flatten output with transaction details (tx_hash, epoch_no, block_height, block_time) instead of tx_hashes array #239
  • Output - /epoch_params - Update cost_models to JSON (upstream change in node) #239
  • Output - /account_assets , /address_assets - Flatten the output result (instead of asset_list array) making it easier to apply horizontal filtering based on any of the fields #239
  • Output - Align output fields for /account_utxos , /address_utxos, /asset_utxos , /script_utxos and /utxo_info to return same schema giving complete details about UTxOs involved, with few fields toggled based on extended input flag #239
  • Output - /pool_list - Add various details to the endpoint for each pool, and expand pool_list to include retired pools as well. (pool_id_hex,active_epoch_no,margin,fixed_cost,pledge,reward_addr,owners,relays,ticker,meta_url,meta_hash,pool_status,retiring_epoch) - this should mean some of the requests to pool_info should no longer be required #239
  • Output - /pool_updates - In v0, pool_updates only provided pool registration updates, while pool_status corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have update_type (enum of either registration or deregistration) instead of pool_status. As a side-effect, since a registration transaction only has retiring_epoch as metadata, all the other fields will show up as null for such a transaction #239
  • Output - /pool_metadata , /pool_relays - Add pool_status field to denote whether pool is retired #239
  • Output - /datum_info - Rename hash to datum_hash and add creation_tx_hash #239
  • Output - /native_script_list - Remove script column (as it has pretty large output better queried against script_info), add size and change type to text #239
  • Output - /plutus_script_list - Add type and size to output #239
  • Output - /asset_info - Add cip68_metadata JSONB field #239
  • Output - /pool_history - Add member_rewards #225

Chores:

  • Update base version to v1 from v0 ( cardano-community/guild-operators#1690 )
  • Allow Bearer Authentication against endpoints #239
  • Cron job to apply corrections to epoch info #239
  • epoch_info_cache Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239
  • Make use of new consumed_by_tx_in_id column in tx_out from dbsync 13.1.1.3 across endpoints.
  • Fix _last_active_stake_validated_epoch in active_stake_cache #222
  • Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223
  • Move control_table entry in cache tables to the end (instead of start) #226
  • Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226
  • Update SQLs as per SQLFluff linting guidelines #226
  • Fix for tip check in cron jobs #217
  • Update cron jobs to exit if the database has not received block in 5 mins #200
  • Update active stake cache to use control table #196
  • Update Asset Info Cache entry whenever asset registry cache has an update #194
  • Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177
  • Swap grestrpcs file to list exceptions and treat everything else as RPC ( cardano-community/guild-operators#1690 )
  • Update grest-poll.sh to have stricter spec validation and add health check for asset_registry ( cardano-community/guild-operators#1690 )
  • Update guild-deploy.sh to include latest pre-release for ogmios ( cardano-community/guild-operators#1690 )
  • Make use of asset-txo-cache for top assets on mainnet, and use this cache for asset_addresses and policy_asset_addresses #250
  • Add v0 RPC redirectors to keep serve v0 endpoints from v1 #250
  • Convert few simple RPC functions from PLPGSQL to SQL language to help with inline filtering #250
  • Address linting results from SQLFluff #250
  • Move db-scripts from guild-operators repository to koios-artifacts repository #250
  • Drop stale db-scripts/genesis_table.sql file #250
  • Add 3 additional indexes for collateral and reference inputs based on query times #250
  • Add top 3 assets for preview/preprod to asset-txo-cache #250
  • Bump schema version for koios-1.1.0 #249

Full Changelog can be found here for Specs and SQL queries, and here for scripts.

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.0 - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

  1. Update your node, dbsync binaries and ogmios. You can use the sample below as reference (change network and top folder according to your environment):
./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.0 -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

  1. Shutdown your node and dbsync services, and checkout updated dbsync release - and start node back up (replace cnode as per your environment in instructions):
sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
cd ~/git/cardano-db-sync
git fetch
git checkout 13.1.1.3
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were shutdown. You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

  1. Wait for your node to be in sync (you can monitor via gLiveView), and then start dbsync using sudo systemctl start cnode-dbsync). The new release of dbsync will go through and migrate all your old entries to include additional information - the timeline for this is going to be closer to doing a full sync, thus - if you prefer, now would be a good time to reset your DB altogether, update postgres versions, etc. We'd recommend Postgres 15 and above, but it's up to individuals. Wait for dbsync to finish adapting data, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

  2. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.0 -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

If instead you're an existing instance provider using koios-1.1.0rc, the update process would be:

  1. Update setup-grest.sh:
./setup-grest.sh -b koios-1.1.0 -r

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask in the group above.

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @ray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @maxee , @Jack-0

Contributors

rcmorano, robinboening, and 29 other contributors
Assets 2
Loading