Koios 1.1.0rc

Koios 1.1.0rc

v1.1.0rc a release candidate, but will still be open for public testing for a week or two (to include any fixes we missed to check to be added to the API).

This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (/api/v1).
The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Also, you'd see quite a few new endpoint additions below, that'd be helping out with slightly lighter version of queries. To keep migration paths easier, we will ensure both v0 and v1 versions of the release is up for a month post release, before retiring v0.

While we understand there is still some scope for improvement especially on asset query performance (though you should see improvements in asset_info already, alongwith easier filtering on *_utxos), that will be one of our primary focus points on upcoming Koios Lite project.

For instance providers, since the underlying infrastructure would've changed drastically - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	17-10-2023 22:00 UTC	https://guild.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
Preview	18-10-2023 22:00 UTC	https://preview.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
PreProd	22-10-2023 22:00 UTC	https://preprod.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
Mainnet	25-10-2023 22:00 UTC	https://api.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2

---

Changes for API

Expand

Deprecations

• /tx_utxos - No longer required as replaced by /utxo_info #227

New endpoints added

• /pool_registrations - List of all pool registrations initiated in the requested epoch #239
• /pool_retirements - List of all pool retirements initiated in the requested epoch #239
• /treasury_withdrawals - List of withdrawals made from treasury #239
• /reserve_withdrawals - List of withdrawals made from reserves (MIRs) #239
• /account_txs - Transactions associated with a given stake address #239
• /address_utxos - Get UTxO details for requested addresses #239
• /asset_utxos - Get UTxO details for requested assets #239
• /script_utxos - Get UTxO details for requested script hashes #239
• /utxo_info - Details for requested UTxO arrays #239
• /script_info - Information about a given script FROM script hashes #239
• /ogmios - Expose Ogmios endpoints ( cardano-community/guild-operators#1690 )

Data Input/Output Changes

• Input - /account_utxos , /credential_utxos - Accept extended as an additional flag - which enables asset_list, reference_script and inline_datum to the output #239
• Output - /block_txs - Flatten output with transaction details (tx_hash, epoch_no, block_height, block_time) instead of tx_hashes array #239
• Output - /epoch_params - Update cost_models to JSON (upstream change in node) #239
• Output - /account_assets , /address_assets - Flatten the output result (instead of asset_list array) making it easier to apply horizontal filtering based on any of the fields #239
• Output - Align output fields for /account_utxos , /address_utxos, /asset_utxos , /script_utxos and /utxo_info to return same schema giving complete details about UTxOs involved, with few fields toggled based on extended input flag #239
• Output - /pool_list - Add various details to the endpoint for each pool, and expand pool_list to include retired pools as well. (pool_id_hex,active_epoch_no,margin,fixed_cost,pledge,reward_addr,owners,relays,ticker,meta_url,meta_hash,pool_status,retiring_epoch) - this should mean some of the requests to pool_info should no longer be required #239
• Output - /pool_updates - In v0, pool_updates only provided pool registration updates, while pool_status corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have update_type (enum of either registration or deregistration) instead of pool_status. As a side-effect, since a registration transaction only has retiring_epoch as metadata, all the other fields will show up as null for such a transaction #239
• Output - /pool_metadata , /pool_relays - Add pool_status field to denote whether pool is retired #239
• Output - /datum_info - Rename hash to datum_hash and add creation_tx_hash #239
• Output - /native_script_list - Remove script column (as it has pretty large output better queried against script_info), add size and change type to text #239
• Output - /plutus_script_list - Add type and size to output #239
• Output - /asset_info - Add cip68_metadata JSONB field #239
• Output - /pool_history - Add member_rewards #225

Chores:

• Update base version to v1 from v0 ( cardano-community/guild-operators#1690 )
• Allow Bearer Authentication against endpoints #239
• Cron job to apply corrections to epoch info #239
• epoch_info_cache Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239
• Make use of new consumed_by_tx_in_id column in tx_out from dbsync 13.1.1.3 across endpoints.
• Fix _last_active_stake_validated_epoch in active_stake_cache #222
• Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223
• Move control_table entry in cache tables to the end (instead of start) #226
• Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226
• Update SQLs as per SQLFluff linting guidelines #226
• Fix for tip check in cron jobs #217
• Update cron jobs to exit if the database has not received block in 5 mins #200
• Update active stake cache to use control table #196
• Update Asset Info Cache entry whenever asset registry cache has an update #194
• Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177
• Swap grestrpcs file to list exceptions and treat everything else as RPC ( cardano-community/guild-operators#1690 )
• Update grest-poll.sh to have stricter spec validation and add health check for asset_registry ( cardano-community/guild-operators#1690 )
• Update guild-deploy.sh to include latest pre-release for ogmios ( cardano-community/guild-operators#1690 )

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.0rc - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

1. Update your node, dbsync binaries and ogmios. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.0rc -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services, and checkout updated dbsync release - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
cd ~/git/cardano-db-sync
git fetch
git checkout 13.1.1.3
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were shutdown. You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. Wait for your node to be in sync (you can monitor via gLiveView), and then start dbsync using sudo systemctl start cnode-dbsync). The new release of dbsync will go through and migrate all your old entries to include additional information - the timeline for this is going to be closer to doing a full sync, thus - if you prefer, now would be a good time to reset your DB altogether, update postgres versions, etc. We'd recommend Postgres 15 and above, but it's up to individuals. Wait for dbsync to finish adapting data, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

4. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.0rc -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @ray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @maxee , @Jack-0