v1.2.0

Koios 1.2.0

This is a finalised release that builds on 1.2.0a to provide support for CIP-129 and add a summary of votes for given proposal. The changes accordingly are primarily only targetting Governance endpoints. While we do not have object types created on different testnet eras, our testing and specs are accordingly primitive at best, and we expect a follow-up release once there is enough participation.
Please go through the changelogs below (and if you werent using koios-1.2.0a, please go through previous changelogs as well):

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	30-08-2024 10:00 UTC	https://guild.koios.rest/	v1.2.0	koios-1.2.0	13.5.0.1	9.1.1
Preview	30-08-2024 10:00 UTC	https://preview.koios.rest/	v1.2.0	koios-1.2.0	13.5.0.1	9.1.1
PreProd	30-08-2024 14:00 UTC	https://preprod.koios.rest/	v1.2.0	koios-1.2.0	13.5.0.1	9.1.1
Mainnet*	31-08-2024 15:30 UTC	https://api.koios.rest/	v1.2.0	koios-1.2.0	13.5.0.1	9.1.1

---

Changes for API

Expand

New endpoints added:

• /proposal_voting_summary - Get a summary of votes cast on specified governance action [#300]

Data Input/Output Changes:

• Input - /commitee_votes - Will require _cc_hot_id which will accept committee member hot key formatted in bech32 as per CIP-0005/129 [#300]
• Input - /voter_proposal_list - Will require _voter_id which will accept DRep/SPO/Committee member formatted in bech32 as per CIP-0005/129 [#300]
• Input - /proposal_votes - Will require _proposal_id which will accept government proposal ID formatted in bech32 as per CIP-129 [#300]
• Output - /drep_metadata , /drep_updates, - added column has_script which shows if given credential is a script hash [#300]
• Output - /drep_votes , /proposal_list , /committee_info - added column proposal_id to show proposal action ID in accordance with CIP-129 [#300]
• Output - /proposal_votes , - voter is renamed to voter_id and shows DRep/Pool/Committee member formatted in bech32 as per CIP-129 [#300]
• Output - Any references to drep in output columns is now assumed to be in CIP-129 format [#300]

Deprecations:

• None

Retirements:

• None

Chores:

• Change indexing for dreps from view to hex [#300]
• Extend utility functions for CIP-129 conversions from hex [#300]

Full Changelog can be found here for SQL queries and Specs , and here for scripts used.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.2.0

The update process as usual would be:

1. Update your node, submitapi, dbsync and ogmios binaries. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.2.0 -s plfdom

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were against node being in shutdown state (or use mithril-client or use https://csnapshots.io). You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. While you Wait for your node to be in sync (you can monitor via gLiveView), you can proceed with dropping current dbsync DB and restoring from snapshot. To do so, use the below (execute commands one by one - verifying output):

We'd recommend using Postgres 16.3 and above, but it's up to individuals.

cd ~/git/cardano-db-sync
git fetch
git checkout 13.5.0.1
./scripts/postgresql-setup.sh --recreatedb
rm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state
ln -s ~/git/cardano-db-sync/schema ${CNODE_HOME}/guild-db/schema 
cd $CNODE_HOME/scripts ; ./dbsync.sh -d

If using snapshot below, please delete the line containing error-on-exit in scripts/postgresql-setup.sh (restore-snapshot function), as there are expected errors for missing roles/schemas that are safe to ignore while restoring from snapshot below.

If you're on mainnet, you'd likely want to restore from snapshot:

wget 'https://share.koios.rest/api/public/dl/xFdZDfM4/dbsync/mainnet-dbsnap-epoch505-x86_64.tgz' -O /tmp/dbsyncsnap.tgz
scripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state

Once the above is complete, you can start dbsync by sudo systemctl start cnode-dbsync. Wait for dbsync to finish synching, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take up to 2 days).

5. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.2.0 -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @xray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @maxee , @Jack-0, @larestrepo

v1.2.0a

Koios 1.2.0a

This release marks initial support for Conway era , with special impetus on Governance data. It brings with it 14 new endpoints enhancing ways to fetch data pertaining to new governance objects. Also, based on community feedback - it includes a minor breaking change (to block_tx_info and tx_info - to whitelist flags desired in output). Also, note that this release is based off cardano-db-sync 13.4.0.2 and node 9.1.0

This release is marked as alpha, as the governance artefacts on Sanchonet are incomplete and there are open questions raised against teams around significance/validation of some fields. Based on inputs we receive, we will follow-up a final version that may contain some minor updates - primarily on governance endpoints.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	23-08-2024 12:00 UTC	https://guild.koios.rest/	v1.2.0a	koios-1.2.0a	13.4.0.2	9.1.0
Preview	23-08-2024 12:00 UTC	https://preview.koios.rest/	v1.2.0a	koios-1.2.0a	13.4.0.2	9.1.0
PreProd	23-08-2024 13:00 UTC	https://preprod.koios.rest/	v1.2.0a	koios-1.2.0a	13.4.0.2	9.1.0
Mainnet*	30-08-2024 08:00 UTC	https://api.koios.rest/	v1.2.0a	koios-1.2.0a	13.4.0.2	9.1.0

---

Changes for API

Expand

New endpoints added:

• /tx_cbor - Raw transaction CBOR against a transaction [#298]
• /drep_epoch_summary - Summary of voting power and DRep count for each epoch [#298]
• /drep_list - List of all active delegated representatives (DReps) [#298]
• /drep_info - Get detailed information about requested delegated representatives (DReps) [#298]
• /drep_metadata - List metadata for requested delegated representatives (DReps) [#298]
• /drep_updates - List of updates for requested (or all) delegated representatives (DReps) [#298]
• /drep_votes - List of all votes casted by requested delegated representative (DRep) [#298]
• /drep_delegators - List of all delegators to requested delegated representative (DRep) [#298]
• /committee_info - Information about active committee and its members [#298]
• /committee_votes - List of all votes casted by given committee member or collective [#298]
• /proposal_list - List of all governance proposals [#298]
• /voter_proposal_list - List of all governance proposals for specified DRep, SPO or Committee credential [#298]
• /proposal_votes - List of all votes cast on specified governance action [#298]
• /pool_votes - List of all votes casted by a pool [#298]

Data Input/Output Changes:

• Input - /block_tx_info, /tx_info - collateral_tx_out -> asset_list - Outputs for collateral tx out are never created on-chain and thus, cannot be queried from ma_tx_out. Instead a rough description of assets involved are saved , which is now returned as info from ledger. This is returned as-is from dbsync and does not adhere to asset_list schema we use in other endpoints. [#298]
• Input - /tx_info , /block_tx_info - These endpoints now require you to specify what all information you'd like to retrieve from a transaction, providing flags _inputs , _metadata, _assets , _withdrawals, _certs, _scripts, _bytecode, _governance [#298]
• Output - /policy_asset_mints , /policy_asset_info, /asset_info - Will return latest mint transaction that has metadata (instead of latest mint transaction) details (excluding burn transactions) [#298]
• Output - /account_info , /pool_info , /pool_list - Add deposit field to output for deposit associated with registration [#298]
• Output - /account_info - Add delegated_drep field to the output [#298]
• Output - /block_tx_info , /tx_info - Add treasury_deposit, voting_procedures and proposal_procedures to the output [#298]
• Output - /epoch_params - Add various fields to epoch_params as per Conway protocol parameters [#298]
• Output - /pool_metadata, /pool_relays - Remove pool_status field from output (it's already listed in pool_info and list) [#298]
• Output - /pool_updates - owners is now a JSONB field instead of JSONB array [#298]

Deprecations:

• None

Retirements:

• None

Chores:

• Remove unused info from asset_info_cache - first_mint_tx_id , first_mint_keys , last_mint_keys are not used/required [#286]
• Add last_mint_meta_tx_id field to asset_info_cache - to return latest asset that does have metadata [#286]
• Reduce redundant cache information for pool stake as we now only retain 3 epochs in pool_active_stake_cache as the rest is already in pool_history_cache [#289]
• Retire v0 SQL files (endpoints were already removed) from repository [#286]
• Overwrite next epoch once on every execution (this is to avoid nonce mismatch if calculated too early from node) [#286]
• Reduce reliance on pool_info_cache where possible to query live metadata [#298]
• Make use of pool_stat instead of epoch_stake for pool_history_cache [#294]
• instant_reward table in dbsync moved to reward_rest
• ada_pots : deposit now split into three different types of deposits

Full Changelog can be found here for SQL queries and Specs , and here for scripts used.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.2.0a

The update process as usual would be:

1. Update your node, submitapi, dbsync and ogmios binaries. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.2.0a -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were against node being in shutdown state (or use mithril-client or use https://csnapshots.io). You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. While you Wait for your node to be in sync (you can monitor via gLiveView), you can proceed with dropping current dbsync DB and restoring from snapshot. To do so, use the below (execute commands one by one - verifying output):

We'd recommend using Postgres 16 and above, but it's up to individuals.

cd ~/git/cardano-db-sync
git fetch
git checkout 13.4.0.2
./scripts/postgresql-scripts.sh --recreatedb
rm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state
ln -s ~/git/cardano-db-sync/schema ${CNODE_HOME}/guild-db/schema 
cd $CNODE_HOME/scripts ; ./dbsync.sh -d

If you're on mainnet, you'd likely want to restore from snapshot:

wget https://share.koios.rest/api/public/dl/xFdZDfM4%2Fdbsync%2Fmainnet-dbsnap-epoch504-x86_64.tgz -O /tmp/dbsyncsnap.tgz
scripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state

Once the above is complete, you can start dbsync by sudo systemctl start cnode-dbsync. Wait for dbsync to finish synching, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take up to 2 days).

5. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.2.0a -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @ho...

v1.1.2

Koios 1.1.2

This release is minor bugfix for data consistency changes behind the scenes. It has no impact to any of the API endpoints.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	21-05-2024 22:00 UTC	https://guild.koios.rest/	v1.1.2	koios-1.1.2	13.2.0.2	8.9.2
Preview	21-05-2024 22:00 UTC	https://preview.koios.rest/	v1.1.2	koios-1.1.2	13.2.0.2	8.9.2
PreProd	21-05-2024 22:00 UTC	https://preprod.koios.rest/	v1.1.2	koios-1.1.2	13.2.0.2	8.9.2
Mainnet*	22-05-2024 08:00 UTC	https://api.koios.rest/	v1.1.2	koios-1.1.2	13.2.0.2	8.9.2

---

Changes for API

Expand

New endpoints added:

• None

Data (Output only) Changes:

• None

Deprecations:

• None

Retirements:

• None

Chores:

• Performance optimisation for pool_info : Add a pool_delegators_list RPC that brings across only minimal information required by pool_info, thereby improving it's performance #281
• Fix pool_history stats for latest epochs by restricting cache to current - 3 epoch while calculating the subsequent information using live status from the database #282
• Fix rewards/treasury/reserves calculation in account_info , account_info_cached and stake_distribution_cache.

Full Changelog can be found here for SQL queries and Specs , and here for scripts used.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.2 - for which you can visit the SQL queries used here.

If you're on koios-1.1.1, all you'd need to do is run ./setup-grest.sh -b koios-1.1.2 -r and wait for cache to be populated. In case you're on an older release, the update process would be:

1. Update your node, submitapi, dbsync and ogmios binaries. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.2 -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were against node being in shutdown state (or use mithril-client or use https://csnapshots.io). You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. While you Wait for your node to be in sync (you can monitor via gLiveView), you can proceed with dropping current dbsync DB and restoring from snapshot. To do so, use the below (execute commands one by one - verifying output):

We'd recommend using Postgres 15 and above, but it's up to individuals.

cd ~/git/cardano-db-sync
git fetch
git checkout 13.2.0.2
./scripts/postgresql-scripts.sh --recreatedb
rm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state
ln -s ~/git/cardano-db-sync/schema ${CNODE_HOME}/guild-db/schema 
cd $CNODE_HOME/scripts ; ./dbsync.sh -d

If you're on mainnet, you'd likely want to restore from snapshot:

wget https://snapshots.koios.rest/dbsync-13.2-mainnet/db-sync-snapshot-schema-13.2-epoch-482-aarch64.tgz -O /tmp/dbsyncsnap.tgz
scripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state

Once the above is complete, you can start dbsync by sudo systemctl start cnode-dbsync. Wait for dbsync to finish synching, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

5. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.2 -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @xray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @maxee , @Jack-0

Koios v1.1.1

Koios 1.1.1

This release primarily focuses on backend performance fixes and work with updates from node-8.9.2 and dbsync 13.2.0.2 - we have also started preparing compatibility with upcoming koios lite release, to make a seamless swap for specific endpoints without any impact to consumers. There are no breaking (impact to existing columns or inputs) changes with this release, but we dp have retired 2 deprecated endpoints that were almost unused on mainnet.

Due to the amount of backend changes in queries, there is a chance that we may have missed some data accuracy checks, and - hence - would like to test in non-mainnet networks first before marking final release. Accordingly, any testing/reports of data inconsistency would be welcome.

For instance providers, since there is major version bump of underlying infrastructure components (you need to re-create DB when updating to 13.2.0.2 if coming from 13.1.x.x) - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	04-05-2024 22:00 UTC	https://guild.koios.rest/	v1.1.1	koios-1.1.1	13.2.0.2	8.9.2
Preview	05-05-2024 22:00 UTC	https://preview.koios.rest/	v1.1.1	koios-1.1.1	13.2.0.2	8.9.2
PreProd	05-05-2024 22:00 UTC	https://preprod.koios.rest/	v1.1.1	koios-1.1.1	13.2.0.2	8.9.2
Mainnet*	11-05-2024 22:00 UTC	https://api.koios.rest/	v1.1.1	koios-1.1.1	13.2.0.2	8.9.2

---

Changes for API

Expand

New endpoints added:

• /policy_asset_mints - List of mint/burn count for all assets minted under a policy #269
• /block_tx_info - Equivalent of tx_info but uses blocks as inputs to fetch tx_info against all tx in the block[s] requested, also contains additional flags to control performance and output #255
• /cli_protocol_params - Return protocl-parameters as returned by cardano-cli from Koios servers #269

Data (Output only) Changes:

• /reserve_withdrawals , /treasury_withdrawals - Add earned_epoch and spendable_epoch fields #269
• /block - Add parent_hash field #263
• /account_list - Add stake_address_hex and script_hash fields #263
• /asset_list - Add script_hash field #263
• /asset_summary - Add addresses field #263
• /asset_addresses , /asset_nft_address and /policy_asset_addresses - Add stake_address field #262
• Fix /script_utxos as it was incorrectly returning object instead of array for asset_list #269
• /tx_info - Add plutus_contract -> spends_input to plutus_contracts to point the input transaction being consumed by the script #269

Deprecations:

• None

Retirements:

• asset_address_list and asset_policy_info endpoints are now retired, as they were marked as deprecated in Koios 1.0.10 , and we have seen it's usage to be negligible (only a single hit in 48 hours on mainnet while marking this release). #269

Chores:

• Retire stake_distribution_new_accounts and stake_snapshot_cache cache, as we directly perform lookup on live tables for newly registered accounts #269
• Active stake cache no longer reads the logs, but instead relies on newly added epoch_sync_progress table #269
• Reduce asset_info_cache rollback lookup from 1000 to 250 #269
• Replace consumed_by_tx_in_id references in SQL by consumed_by_tx_id #269
• pool_history_cache now breaks into populating 500 epochs at a time (on guildnet, this query used to run for hours against ~20K epochs) #269
• Accomodate splitting of reward table into instant_reward #269
• Add a check in stake_distribution_cache to ensure that epoch info cache was run for current - 1 epoch. #269
• Change return type for internal function grest.cip67_strip_label from text to bytea #269
• Remove any references to tx_in as it is no longer required #269
• Remove references to pool_offline_data with off_chain_pool_data #269
• Disable running asset-txo-cache-update as the endpoints leveraging asset-txo will be moved to koios-lite
• Convert block, account_list, asset_list asset_token_registry from view to function #263
• asset_info_cache - ensure mint tx is only against a positive mint quantity #262
• Include burnt asset transactions in asset_txs #269
• tx_info - Fix spend_redeemers CTE Join condition #269

Full Changelog can be found here for SQL queries and Specs , and here for scripts used.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.1 - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

1. Update your node, submitapi, dbsync and ogmios binaries. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.1 -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were against node being in shutdown state (or use mithril-client or use https://csnapshots.io). You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. While you Wait for your node to be in sync (you can monitor via gLiveView), you can proceed with dropping current dbsync DB and restoring from snapshot. To do so, use the below (execute commands one by one - verifying output):

We'd recommend using Postgres 15 and above, but it's up to individuals.

cd ~/git/cardano-db-sync
git fetch
git checkout 13.2.0.2
./scripts/postgresql-scripts.sh --recreatedb
rm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state
ln -s ~/git/cardano-db-sync/schema ${CNODE_HOME}/guild-db/schema 
cd $CNODE_HOME/scripts ; ./dbsync.sh -d

If you're on mainnet, you'd likely want to restore from snapshot:

wget https://snapshots.koios.rest/dbsync-13.2-mainnet/db-sync-snapshot-schema-13.2-block-10045937-aarch64.tgz -O /tmp/dbsyncsnap.tgz
scripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state

Once the above is complete, you can start dbsync by sudo systemctl start cnode-dbsync. Wait for dbsync to finish synching, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

5. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.1 -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @xray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @m...

Koios 1.1.1rc

Koios 1.1.1rc

v1.1.1rc is a release candidate, but will still be open for public testing for a couple of weeks (to include any fixes we missed to check to be added to the API).

This release primarily focuses on backend performance fixes and work with updates from node-8.7.3 and dbsync 13.2.0.1 - we have also started preparing compatibility with upcoming koios lite release, to make a seamless swap for specific endpoints without any impact to consumers. There are no breaking (impact to existing columns or inputs) changes with this release, but we dp have retired 2 deprecated endpoints that were almost unused on mainnet.

Due to the amount of backend changes in queries, there is a chance that we may have missed some data accuracy checks, and - hence - would like to test in non-mainnet networks first before marking final release. Accordingly, any testing/reports of data inconsistency would be welcome.

For instance providers, since there is major version bump of underlying infrastructure components (you need to re-create DB when updating to 13.2.0.1 if coming from 13.1.x.x) - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	17-03-2024 22:00 UTC	https://guild.koios.rest/	v1.1.1rc	koios-1.1.1rc	13.2.0.2	8.9.2
Preview	20-03-2024 22:00 UTC	https://preview.koios.rest/	v1.1.1rc	koios-1.1.1rc	13.2.0.2	8.9.2
PreProd	20-03-2024 22:00 UTC	https://preprod.koios.rest/	v1.1.1rc	koios-1.1.1rc	13.2.0.2	8.9.2
Mainnet*	10-05-2024 22:00 UTC	https://api.koios.rest/	v1.1.1	koios-1.1.1	13.2.0.2	8.9.2

*For mainnet , we will go for finalised release (instead of v1.1.1rc, but that hasnt been tagged yet - will depend on outcomes of testing)

---

Changes for API

Expand

New endpoints added:

• /asset_policy_mints - List of mint/burn count for all assets minted under a policy #269
• /block_tx_info - Equivalent of tx_info but uses blocks as inputs to fetch tx_info against all tx in the block[s] requested, also contains additional flags to control performance and output #255

Data (Output only) Changes:

• /reserve_withdrawals , /treasury_withdrawals - Add earned_epoch and spendable_epoch fields #269
• /block - Add parent_hash field #263
• /account_list - Add stake_address_hex and script_hash fields #263
• /asset_list - Add script_hash field #263
• /asset_summary - Add addresses field #263
• /asset_addresses , /asset_nft_address and /policy_asset_addresses - Add stake_address field #262

Deprecations:

• None

Retirements:

• asset_address_list and asset_policy_info endpoints are now retired, as they were marked as deprecated in Koios 1.0.10 , and we have seen it's usage to be negligible (only a single hit in 48 hours on mainnet while marking this release). #269

Chores:

• Retire stake_distribution_new_accounts and stake_snapshot_cache cache, as we directly perform lookup on live tables for newly registered accounts #269
• Active stake cache no longer reads the logs, but instead relies on newly added epoch_sync_progress table #269
• Reduce asset_info_cache rollback lookup from 1000 to 250 #269
• Replace consumed_by_tx_in_id references in SQL by consumed_by_tx_id #269
• pool_history_cache now breaks into populating 500 epochs at a time (on guildnet, this query used to run for hours against ~20K epochs) #269
• Accomodate splitting of reward table into instant_reward #269
• Add a check in stake_distribution_cache to ensure that epoch info cache was run for current - 1 epoch. #269
• Change return type for internal function grest.cip67_strip_label from text to bytea #269
• Remove any references to tx_in as it is no longer required #269
• Remove references to pool_offline_data with off_chain_pool_data #269
• Disable running asset-txo-cache-update as the endpoints leveraging asset-txo will be moved to koios-lite
• Convert block, account_list, asset_list asset_token_registry from view to function #263
• asset_info_cache - ensure mint tx is only against a positive mint quantity #262

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.1rc - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

1. Update your node, submitapi, dbsync and ogmios binaries. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.1rc -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were against node being in shutdown state (or use mithril-client or use https://csnapshots.io). You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. While you Wait for your node to be in sync (you can monitor via gLiveView), you can proceed with dropping current dbsync DB and restoring from snapshot. To do so, use the below (execute commands one by one - verifying output):

We'd recommend using Postgres 15 and above, but it's up to individuals.

cd ~/git/cardano-db-sync
git fetch
git checkout 13.2.0.1
./scripts/postgresql-scripts.sh --recreatedb
rm -rf ${CNODE_HOME}/guild-db/ledger-state ; mkdir -p ${CNODE_HOME}/guild-db/ledger-state
ln -s ~/git/cardano-db-sync/schema ${CNODE_HOME}/guild-db/schema 
cd $CNODE_HOME/scripts ; ./dbsync.sh -d

If you're on mainnet, you'd likely want to restore from snapshot:

wget https://snapshots.koios.rest/dbsync-13.2-mainnet/db-sync-snapshot-schema-13.2-block-10045937-aarch64.tgz -O /tmp/dbsyncsnap.tgz
scripts/postgresql-setup.sh --restore-snapshot /tmp/dbsyncsnap.tgz ${CNODE_HOME}/guild-db/ledger-state

Once the above is complete, you can start dbsync by sudo systemctl start cnode-dbsync. Wait for dbsync to finish synching, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

5. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.1rc -r -i prmcd
sudo systemctl restart cnode-postgrest cnode-haproxy

Due to difference in script provisioning versions, it is possible that you might have trouble starting cnode-postgrest - if so, check and ensure that you can access $CNODE_HOME/priv/grest.conf file as authenticator user - you can use command as test : sudo -u authenticator cat /opt/cardano/gnode/priv/grest.conf

Since we're again re-building cache, expect the cache to take a bit (few minutes on non-mainnet, while a few hours on mainnet).

---

Other Notes

Expand

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
We're open to have meetings to work together as well as discuss future additions, feel free to ask

Contributors

We are blessed to have community who contribute in various ways, be it by serving community instances, library implementations, testing, or feedbacks. Special thanks to all the contributors that have been helping us improve the query layer along the way
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @reqlez , @huths0lo , @chadle-git , @gufmar , @dostrelith678, @ray-wallet, @edridudi , @mkungla , @xray-robot , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @AlexDochioiu , @DCOneCrypto , @PatrickTobler , @agaffney , @rcmorano , @gitmachtl, @Crypto2099, @robinboening, @fallen-icarus , @M4rc0Russ0 , @HT-Moh , @ducpm2303 , @michele-nuzzi , @maxee , @Jack-0

Koios 1.1.0

Koios 1.1.0

You can refer to here if looking for difference between koios-1.1.0 and koios-1.1.0rc, mostly chores behind-the-scenes

Koios v1.1.0 is the first major [breaking] release for Koios consumers in a while, and is rolled out under new base prefix (/api/v1).
The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. To keep migration paths easier, we have added some v0 re-directors to ensure v0 applications do not break, but users are absolutely recommended to upgrade from deprecated v0, which could be retired in future months (likely with next breaking release at Conway).

While we understand there is still some scope for improvement especially on asset query performance (though you should see quite some improvements already, alongwith easier filtering on *_utxos), that will be one of our primary focus points on upcoming Koios Lite project.

For instance providers, unless coming from v1.1.0rc, the underlying infrastructure would've changed drastically - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	24-11-2023 22:00 UTC	https://guild.koios.rest/	v1.1.0	koios-1.1.0	13.1.1.3	8.1.2
Preview	28-11-2023 22:00 UTC	https://preview.koios.rest/	v1.1.0	koios-1.1.0	13.1.1.3	8.1.2
PreProd	08-12-2023 22:00 UTC	https://preprod.koios.rest/	v1.1.0	koios-1.1.0	13.1.1.3	8.1.2
Mainnet	13-12-2023 22:00 UTC	https://api.koios.rest/	v1.1.0	koios-1.1.0	13.1.1.3	8.1.2

---

Changes for API (from v1.0.11)

Expand

Deprecations

• /tx_utxos - No longer required as replaced by /utxo_info #227

New endpoints added

• /pool_registrations - List of all pool registrations initiated in the requested epoch #239
• /pool_retirements - List of all pool retirements initiated in the requested epoch #239
• /treasury_withdrawals - List of withdrawals made from treasury #239
• /reserve_withdrawals - List of withdrawals made from reserves (MIRs) #239
• /account_txs - Transactions associated with a given stake address #239
• /address_utxos - Get UTxO details for requested addresses #239
• /asset_utxos - Get UTxO details for requested assets #239
• /script_utxos - Get UTxO details for requested script hashes #239
• /utxo_info - Details for requested UTxO arrays #239
• /script_info - Information about a given script FROM script hashes #239
• /ogmios - Expose Ogmios endpoints ( cardano-community/guild-operators#1690 )

Data Input/Output Changes

• Input - /account_utxos , /credential_utxos - Accept extended as an additional flag - which enables asset_list, reference_script and inline_datum to the output #239
• Output - /block_txs - Flatten output with transaction details (tx_hash, epoch_no, block_height, block_time) instead of tx_hashes array #239
• Output - /epoch_params - Update cost_models to JSON (upstream change in node) #239
• Output - /account_assets , /address_assets - Flatten the output result (instead of asset_list array) making it easier to apply horizontal filtering based on any of the fields #239
• Output - Align output fields for /account_utxos , /address_utxos, /asset_utxos , /script_utxos and /utxo_info to return same schema giving complete details about UTxOs involved, with few fields toggled based on extended input flag #239
• Output - /pool_list - Add various details to the endpoint for each pool, and expand pool_list to include retired pools as well. (pool_id_hex,active_epoch_no,margin,fixed_cost,pledge,reward_addr,owners,relays,ticker,meta_url,meta_hash,pool_status,retiring_epoch) - this should mean some of the requests to pool_info should no longer be required #239
• Output - /pool_updates - In v0, pool_updates only provided pool registration updates, while pool_status corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have update_type (enum of either registration or deregistration) instead of pool_status. As a side-effect, since a registration transaction only has retiring_epoch as metadata, all the other fields will show up as null for such a transaction #239
• Output - /pool_metadata , /pool_relays - Add pool_status field to denote whether pool is retired #239
• Output - /datum_info - Rename hash to datum_hash and add creation_tx_hash #239
• Output - /native_script_list - Remove script column (as it has pretty large output better queried against script_info), add size and change type to text #239
• Output - /plutus_script_list - Add type and size to output #239
• Output - /asset_info - Add cip68_metadata JSONB field #239
• Output - /pool_history - Add member_rewards #225

Chores:

• Update base version to v1 from v0 ( cardano-community/guild-operators#1690 )
• Allow Bearer Authentication against endpoints #239
• Cron job to apply corrections to epoch info #239
• epoch_info_cache Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239
• Make use of new consumed_by_tx_in_id column in tx_out from dbsync 13.1.1.3 across endpoints.
• Fix _last_active_stake_validated_epoch in active_stake_cache #222
• Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223
• Move control_table entry in cache tables to the end (instead of start) #226
• Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226
• Update SQLs as per SQLFluff linting guidelines #226
• Fix for tip check in cron jobs #217
• Update cron jobs to exit if the database has not received block in 5 mins #200
• Update active stake cache to use control table #196
• Update Asset Info Cache entry whenever asset registry cache has an update #194
• Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177
• Swap grestrpcs file to list exceptions and treat everything else as RPC ( cardano-community/guild-operators#1690 )
• Update grest-poll.sh to have stricter spec validation and add health check for asset_registry ( cardano-community/guild-operators#1690 )
• Update guild-deploy.sh to include latest pre-release for ogmios ( cardano-community/guild-operators#1690 )
• Make use of asset-txo-cache for top assets on mainnet, and use this cache for asset_addresses and policy_asset_addresses #250
• Add v0 RPC redirectors to keep serve v0 endpoints from v1 #250
• Convert few simple RPC functions from PLPGSQL to SQL language to help with inline filtering #250
• Address linting results from SQLFluff #250
• Move db-scripts from guild-operators repository to koios-artifacts repository #250
• Drop stale db-scripts/genesis_table.sql file #250
• Add 3 additional indexes for collateral and reference inputs based on query times #250
• Add top 3 assets for preview/preprod to asset-txo-cache #250
• Bump schema version for koios-1.1.0 #249

Full Changelog can be found here for Specs and SQL queries, and here for scripts.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.0 - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

1. Update your node, dbsync binaries and ogmios. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.0 -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services, and checkout updated dbsync release - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
cd ~/git/cardano-db-sync
git fetch
git checkout 13.1.1.3
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were shutdown. You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. Wait for your node to be in sync (you can monitor via gLiveView), and then start dbsync using `sudo systemctl start cnode-dbs...

Koios 1.1.0rc

Koios 1.1.0rc

v1.1.0rc a release candidate, but will still be open for public testing for a week or two (to include any fixes we missed to check to be added to the API).

This will be first major [breaking] release for Koios consumers in a while, and will be rolled out under new base prefix (/api/v1).
The major work with this release was to start making use of newer flags in dbsync which help performance of queries under new endpoints. Also, you'd see quite a few new endpoint additions below, that'd be helping out with slightly lighter version of queries. To keep migration paths easier, we will ensure both v0 and v1 versions of the release is up for a month post release, before retiring v0.

While we understand there is still some scope for improvement especially on asset query performance (though you should see improvements in asset_info already, alongwith easier filtering on *_utxos), that will be one of our primary focus points on upcoming Koios Lite project.

For instance providers, since the underlying infrastructure would've changed drastically - expect some downtime to work through the update when you get to it.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	17-10-2023 22:00 UTC	https://guild.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
Preview	18-10-2023 22:00 UTC	https://preview.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
PreProd	22-10-2023 22:00 UTC	https://preprod.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2
Mainnet	25-10-2023 22:00 UTC	https://api.koios.rest/	v1.1.0rc	koios-1.1.0rc	13.1.1.3	8.1.2

---

Changes for API

Expand

Deprecations

• /tx_utxos - No longer required as replaced by /utxo_info #227

New endpoints added

• /pool_registrations - List of all pool registrations initiated in the requested epoch #239
• /pool_retirements - List of all pool retirements initiated in the requested epoch #239
• /treasury_withdrawals - List of withdrawals made from treasury #239
• /reserve_withdrawals - List of withdrawals made from reserves (MIRs) #239
• /account_txs - Transactions associated with a given stake address #239
• /address_utxos - Get UTxO details for requested addresses #239
• /asset_utxos - Get UTxO details for requested assets #239
• /script_utxos - Get UTxO details for requested script hashes #239
• /utxo_info - Details for requested UTxO arrays #239
• /script_info - Information about a given script FROM script hashes #239
• /ogmios - Expose Ogmios endpoints ( cardano-community/guild-operators#1690 )

Data Input/Output Changes

• Input - /account_utxos , /credential_utxos - Accept extended as an additional flag - which enables asset_list, reference_script and inline_datum to the output #239
• Output - /block_txs - Flatten output with transaction details (tx_hash, epoch_no, block_height, block_time) instead of tx_hashes array #239
• Output - /epoch_params - Update cost_models to JSON (upstream change in node) #239
• Output - /account_assets , /address_assets - Flatten the output result (instead of asset_list array) making it easier to apply horizontal filtering based on any of the fields #239
• Output - Align output fields for /account_utxos , /address_utxos, /asset_utxos , /script_utxos and /utxo_info to return same schema giving complete details about UTxOs involved, with few fields toggled based on extended input flag #239
• Output - /pool_list - Add various details to the endpoint for each pool, and expand pool_list to include retired pools as well. (pool_id_hex,active_epoch_no,margin,fixed_cost,pledge,reward_addr,owners,relays,ticker,meta_url,meta_hash,pool_status,retiring_epoch) - this should mean some of the requests to pool_info should no longer be required #239
• Output - /pool_updates - In v0, pool_updates only provided pool registration updates, while pool_status corresponded to current status of pool. With v1, we will have registration as well as deregistration transactions, and each transaction will have update_type (enum of either registration or deregistration) instead of pool_status. As a side-effect, since a registration transaction only has retiring_epoch as metadata, all the other fields will show up as null for such a transaction #239
• Output - /pool_metadata , /pool_relays - Add pool_status field to denote whether pool is retired #239
• Output - /datum_info - Rename hash to datum_hash and add creation_tx_hash #239
• Output - /native_script_list - Remove script column (as it has pretty large output better queried against script_info), add size and change type to text #239
• Output - /plutus_script_list - Add type and size to output #239
• Output - /asset_info - Add cip68_metadata JSONB field #239
• Output - /pool_history - Add member_rewards #225

Chores:

• Update base version to v1 from v0 ( cardano-community/guild-operators#1690 )
• Allow Bearer Authentication against endpoints #239
• Cron job to apply corrections to epoch info #239
• epoch_info_cache Remove protocol parameters, as they can be queried from live table. Accordingly update dependent queries #239
• Make use of new consumed_by_tx_in_id column in tx_out from dbsync 13.1.1.3 across endpoints.
• Fix _last_active_stake_validated_epoch in active_stake_cache #222
• Typo for pool_history_cache.sql as well as add a check to ensure epoch_info_cache has run at least once prior to pool_history_cache #223
• Move control_table entry in cache tables to the end (instead of start) #226
• Fix Asset Info Cache (include mint/burn tx rather than sum for meta consideration) #226
• Update SQLs as per SQLFluff linting guidelines #226
• Fix for tip check in cron jobs #217
• Update cron jobs to exit if the database has not received block in 5 mins #200
• Update active stake cache to use control table #196
• Update Asset Info Cache entry whenever asset registry cache has an update #194
• Bump up margin for tx rollback lookup for asset_info_cache to 1000 , as 100 is too small a margin for 2-3 blocks , which can contain more than 100 transactions (of which if oldest transaction contains a mint, it will not get into the cache) #177
• Swap grestrpcs file to list exceptions and treat everything else as RPC ( cardano-community/guild-operators#1690 )
• Update grest-poll.sh to have stricter spec validation and add health check for asset_registry ( cardano-community/guild-operators#1690 )
• Update guild-deploy.sh to include latest pre-release for ogmios ( cardano-community/guild-operators#1690 )

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

---

Instructions for Instance Providers

Expand

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.1.0rc - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.x, the update process would be:

1. Update your node, dbsync binaries and ogmios. You can use the sample below as reference (change network and top folder according to your environment):

./guild-deploy.sh -n $NETWORK -t cnode -b koios-1.1.0rc -s plfdo

Note that the use of f above is to force overwrite your configs , as newer node includes ConwayGenesis placeholder keys. The above command will overwrite your config.json, topology.json as well as user variables, and save them original ones backups, you'd want to copy your user variables into the files modified and re-add port customisations to config.json or topology peering.

2. Shutdown your node and dbsync services, and checkout updated dbsync release - and start node back up (replace cnode as per your environment in instructions):

sudo systemctl stop cnode cnode-dbsync
sudo rm -f /etc/cron.d/cnode-*
cd ~/git/cardano-db-sync
git fetch
git checkout 13.1.1.3
sudo systemctl start cnode

Expect node to take a few hours to startup, you can short-circuit the process by copying $CNODE_HOME/db folder from already upgraded node - while both source and destination node folders were shutdown. You can also check node startup status by viewing node logs (tail -100f $CNODE_HOME/logs/node0.json)

3. Wait for your node to be in sync (you can monitor via gLiveView), and then start dbsync using sudo systemctl start cnode-dbsync). The new release of dbsync will go through and migrate all your old entries to include additional information - the timeline for this is going to be closer to doing a full sync, thus - if you prefer, now would be a good time to reset your DB altogether, update postgres versions, etc. We'd recommend Postgres 15 and above, but it's up to individuals. Wait for dbsync to finish adapting data, you can tail logs via tail -10f $CNODE_HOME/logs/dbsync.json - on mainnet in particular, this could take 2 days).

4. With dbsync updated and on tip, you'd now be ready to go to the next step - updating koios. You'd want to update all components and reset your grest schema, sample command below:

./setup-grest.sh -b koios-1.1.0rc -r -i prmcd
sudo sy...

Koios 1.0.10

Koios 1.0.10

This release primarily focuses on ability to support better DeFi projects alongwith some value addition for existing clients by bringing in 10 new endpoints (paired with 2 deprecations), few additional optional input parameters , and some additional output columns to existing endpoints. The only breaking change/fix is for output returned for tx_info.

Also, dbsync 13.1.0.x has been released and is recommended to be used for this release. It is recommended you shutdown services temporarily while you upgrade (and clear contents of $CNODE_HOME/guild-db/ledger-state), and perhaps better off re-synching from genesis for preview/preprod as the recreation of indexes are not as optimal as re-index against an empty database, specifically from postgres tiling point of view.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	03-03-2023 08:00 UTC	https://guild.koios.rest/	v1.0.10	koios-1.0.10	13.1.0.[0-2]	1.35.[4-6]
Preview	03-03-2023 08:00 UTC	https://preview.koios.rest/	v1.0.10	koios-1.0.10	13.1.0.[0-2]	1.35.[4-6]
PreProd	04-03-2023 08:00 UTC	https://preprod.koios.rest/	v1.0.10	koios-1.0.10	13.1.0.[0-2]	1.35.[4-6]
Mainnet*	12-03-2023 20:00 UTC	https://api.koios.rest/	v1.0.10	koios-1.0.10	13.1.0.[0-2]	1.35.[4-6]

Known Issue(s)

• The pool metadata ticker list might still be missing a few entries - see linked issue on dbsync here
• There is a known issue with datum CBOR mismatch unless synching from scratch, expecting a new release to be marked soon - see linked issue on dbsync here
• Epoch table's Block count is less [by 1] for recent epochs, see linked issue on dbsync here
• [For instance providers only] PostgREST 10.1.x can often result in memory leaks for long-running sessions, can use 9.x or implement auto-restart, underlying issue on hasql here

Changes for API

Deprecations

• /asset_address_list - Renamed to asset_addresses keeping naming line with other endpoints (old endpoint will be retired in future release) #149
• /asset_policy_info - Renamed to policy_asset_info keeping naming line with other endpoints (old endpoint will be retired in future release) #149

New endpoints added

• /asset_addresses - Equivalent of deprecated /asset_address_list #149
• /asset_nft_address - Returns address where the specified NFT sits on #149
• /account_utxos - Returns brief details on non-empty UTxOs associated with a given stake address #149
• /asset_info_bulk - Bulk version of /asset_info #142
• /asset_token_registry - Returns assets registered via token registry on github #145
• /credential_utxos - Returns UTxOs associated with a payment credential #149
• /param_updates - Returns list of parameter update proposals applied to the network #149
• /policy_asset_addresses - Returns addresses with quantity for each asset on a given policy #149
• /policy_asset_info - Equivalent of deprecated /asset_policy_info but with more details in output #149
• /policy_asset_list - Returns list of asset under the given policy (including supply) #142, #149

Data Input/Output Changes

• Input (breaking)- /account_addresses - Add optional _first_only and _empty flags to show only first address with tx or to include empty addresses to output #149 , the change for endpoint consumption is that the default value for _empty is set to false
• Input - /epoch_info - Add optional _include_next_epoch field to show next epoch stats if available (eg: nonce, active stake) #143
• Output (addition) - /account_assets , /address_assets , /address_info, /tx_info, /tx_utxos - Add decimals to output #142
• Output (addition) - /policy_asset_info - Add minting_tx_hash, total_supply, mint_cnt, burn_cnt and creation_time fields to the output #149
• Output (breaking) - /tx_info - Change _invalid_before and _invalid_after to text field #141
• Output (breaking/removal) - tx_info - Remove the field plutus_contracts > [array] > outputs as there is no logic to connect it to inputs spending #163

Chores:

• /epoch_info, /epoch_params - Restrict output to current epoch #149
• /block_info - Use /previous_id field to show previous/next blocks (previously was using block_id/height) #145
• /asset_info/asset_policy_info - Fix mint tx data to be latest #141
• Support new guild scripts revamp #1572
• Add asset token registry check 1606
• New cache table grest.asset_info_cache to hold mint/burn counts alongwith first/last mint tx/keys #142
• Bump to Koios 1.0.10rc #149
• Fix typo in specs for /pool_delegators output column latest_delegation_tx_hash #149
• Add indexes for ones missing after configuring cardano-db-sync 13.1.0.0 #149
• Update PostgREST to be run as authenticator user, whose default statement_timeout is set to 65s and update configs accordingly #1606
• Replace all RPC references for JSON endpoints with JSONB, this allows filtering child members of array elements using cs.[{"key":"value"}] in PostgREST #172
• Fix Asset Info Cache to not rely on being able to decode policy ID (bad data on IO repo) #173
• Handle Pool_list to check metadata entries that have not been populated in pool_offline_data #173
• Fix pool_updates to use pmr_id instead of pool_offline_data.id #173
• Update account_info_cached description to clarify it is effective for registered accounts #173
• Fix asset_info_cache and various address/account/tx endpoints to cater for transactions lost during network congestion #177

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

Instructions for Instance Providers

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.0.10 - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.9, the update process would be executing below:

./setup-grest.sh -b koios-1.0.10 -i pcd -r
sudo systemctl restart cnode-haproxy

If you're upgrading existing koios-1.0.10rc instance(guild/preview networks), the update process would be executing below:

./setup-grest.sh -b koios-1.0.10 -q

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
There are bi-weekly meetings held for development activities on Koios gRest project at every 2nd and 4th Thursday of the month, the details for which can be found in the group above.

Contributors

Special thanks to all the contributors:
@Scitz0 , @rdlrt , @redoracle, @hodlonaut, @gufmar , @edridudi , @mkungla , @nothingalike, @safestak-keith , @QuixoteSystems, @abdelkrimdev, @cardano-apexpool , @reqlez , @huths0lo , @chadle-git

Koios 1.0.10rc

Koios 1.0.10rc

This release primarily focuses on ability to support better DeFi projects alongwith some value addition for existing clients by bringing in 10 new endpoints (paired with 2 deprecations), few additional optional input parameters , and some additional output columns to existing endpoints. The only breaking change/fix is for output returned for tx_info.

Also, dbsync 13.1.0.x has been released and is recommended to be used for this release. It is recommended you shutdown services temporarily while you upgrade, and perhaps better off re-synching from genesis for preview/preprod as the recreation of indexes are not as optimal as fresh-ly created ones for postgres tiling point of view.

This release is meant to be used for non-mainnet (guild,preview and preprod) networks. Mainnet release will be almost the same but tag to be used will be 1.0.10.

API Docs and Schedule for release:

The schedule for monitoring layers to apply update , as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	19-02-2023 08:00 UTC	https://guild.koios.rest/	v1.0.10rc	koios-1.0.10rc	13.1.0.[0-2]	1.35.[4-5]
Preview	19-02-2023 08:00 UTC	https://preview.koios.rest/	v1.0.10rc	koios-1.0.10rc	13.1.0.[0-2]	1.35.[4-5]
PreProd	04-03-2023 08:00 UTC	https://preprod.koios.rest/	v1.0.10rc	koios-1.0.10rc	13.1.0.[0-2]	1.35.[4-5]
Mainnet*	12-03-2023 20:00 UTC	https://api.koios.rest/	v1.0.10	koios-1.0.10	13.1.0.[0-2]	1.35.[4-5]

Mainnet* - The tags for mainnet are dependent on testing, will be tagged in the final week seperately depending on outcome of testing, will not include any breaking changes from v1.0.10rc . The entry is only included here to answer common query

Known Issue(s)

• The pool metadata ticker list might still be missing a few entries - see linked issue on dbsync [here]
• There is a known issue with datum CBOR mismatch, expecting a new release to be marked soon - see linked issue on dbsync here

Changes for API

Deprecations

• /asset_address_list - Renamed to asset_addresses keeping naming line with other endpoints (old endpoint will be retired in future release) #149
• /asset_policy_info - Renamed to policy_asset_info keeping naming line with other endpoints (old endpoint will be retired in future release) #149

New endpoints added

• /asset_addresses - Equivalent of deprecated /asset_address_list #149
• /asset_nft_address - Returns address where the specified NFT sits on #149
• /account_utxos - Returns brief details on non-empty UTxOs associated with a given stake address #149
• /asset_info_bulk - Bulk version of /asset_info #142
• /asset_token_registry - Returns assets registered via token registry on github #145
• /credential_utxos - Returns UTxOs associated with a payment credential #149
• /param_updates - Returns list of parameter update proposals applied to the network #149
• /policy_asset_addresses - Returns addresses with quantity for each asset on a given policy #149
• /policy_asset_info - Equivalent of deprecated /asset_policy_info but with more details in output #149
• /policy_asset_list - Returns list of asset under the given policy (including supply) #142, #149

Data Input/Output Changes

• Input - /account_addresses - Add optional _first_only and _empty flags to show only first address with tx or to include empty addresses to output #149
• Input - /epoch_info - Add optional _include_next_epoch field to show next epoch stats if available (eg: nonce, active stake) #143
• Output (addition) - /account_assets , /address_assets , /address_info, /tx_info, /tx_utxos - Add decimals to output #142
• Output (addition) - /policy_asset_info - Add minting_tx_hash, total_supply, mint_cnt, burn_cnt and creation_time fields to the output #149
• Output (breaking) - /tx_info - Change _invalid_before and _invalid_after to text field #141
• Output (breaking/removal) - tx_info - Remove the field plutus_contracts > [array] > outputs as there is no logic to connect it to inputs spending #163

Chores:

• /epoch_info, /epoch_params - Restrict output to current epoch #149
• /block_info - Use /previous_id field to show previous/next blocks (previously was using block_id/height) #145
• /asset_info/asset_policy_info - Fix mint tx data to be latest #141
• Support new guild scripts revamp #1572
• Add asset token registry check 1606
• New cache table grest.asset_info_cache to hold mint/burn counts alongwith first/last mint tx/keys #142
• Bump to Koios 1.0.10rc #149
• Fix typo in specs for /pool_delegators output column latest_delegation_tx_hash #149
• Add indexes for ones missing after configuring cardano-db-sync 13.1.0.0 #149
• Update PostgREST to be run as authenticator user, whose default statement_timeout is set to 65s and update configs accordingly #1606

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

Instructions for Instance Providers

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.0.10rc - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.9, the update process would be executing below:

./setup-grest.sh -b koios-1.0.10rc -i pcd -r
sudo systemctl restart cnode-haproxy

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
There are bi-weekly meetings held for development activities on Koios gRest project at every 2nd and 4th Thursday of the month, the details for which can be found in the group above.

Koios 1.0.9

Koios 1.0.9

The release introduces changes that are non-breaking for existing GET/POST methods and inputs, but they are breaking for output objects reported using those endpoints.

One of the important behind-the-scenes changes is we've now moved the SQL files used by Koios to koios-artifacts repo instead of guild-operators , this allows us higher flexibility in supporting array of tool versions against koios query versions, and simplifies the Issue/PR-linking across the repositories.

Also, this release starts introducing preview and preprod network support.

Note that as per prior announcements and release notes in v1.0.9rc , the public monitoring instances for Koios legacy testnet instances have now retired as of 18th November 2022. That would not prevent folks to run Koios testnet instance themselves as the scripts still recognize testnet network!!

API Docs and Schedule for release:

The schedule for monitoring layers to apply update, as well as live documentation can be browsed as per table below:

Network	Date	URL	Specs Release	gRest Instance Release	DBSync	Node
GuildNet	18-11-2022 08:00 UTC	https://guild.koios.rest/	v1.0.9	koios-1.0.9	13.0.5	1.35.[3-4]
Preview	18-11-2022 08:00 UTC	https://preview.koios.rest/	v1.0.9	koios-1.0.9	13.0.5	1.35.4
PreProd	18-11-2022 08:00 UTC	https://preprod.koios.rest/	v1.0.9	koios-1.0.9	13.0.5	1.35.[3-4]
Mainnet	19-11-2022 20:00 UTC	https://api.koios.rest/	v1.0.9	koios-1.0.9	13.0.5	1.35.[3-4]
Testnet	< Retired >	-	-	-	-	-

Known Issue(s)

• As part of issue in dbsync 13.0.5 (fixed on master , but not yet part of a release), the pool metadata might be missing a few entries , tho this would typically be for those who might have done multiple updates to their pool metadata, should be sorted with next dbsync release
• (For instance providers only) - cardano-db-sync v13.0.5 will require resync from genesis (see issue here ) to avoid data consistency issue for datum table

Changes for API

New endpoints added

• datum_info - List of datum information for given datum hashes
• account_info_cached - Same as account_info, but serves cached information instead of live one

Data Input/Output changes

• address_info, address_assets, account_assets, tx_info, asset_list asset_summary to align output asset_list object to return array of policy_id, asset_name, fingerprint (and quantity, minting_txs where applicable) #120
• asset_history - Fix metadata to wrap in array to refer to right object #122
• asset_txs - Add optional boolean parameter _history (default: false) to toggle between querying current UTxO set vs entire history for asset #122
• pool_history - fixed_cost, pool_fees, deleg_rewards, epoch_ros will be returned as 0 when null #122
• tx_info - plutus_contracts->outputs can be null #122

Changes for Instance Providers

• SQL queries have been moved from guild-operators repository to koios-artifacts repository. This is to ensure that the updates made to scripts and other tooling do not have a dependency on Koios query versioning #122
• block_info - Use block_no instead of id to check for previous/next block hash #122
• Add topology for preprod and preview networks #122
• setup-grest.sh - Minor bug fixes that caused getmetrics.sh to be not downloaded from github #1575

Full Changelog can be found here for Specs , and here for scripts and SQL queries.

Instructions for Instance Providers

If you're setting up as a new instance provider, you can find instructions here. The tag to use for setup-grest.sh would be koios-1.0.9 - for which you can visit the SQL queries used here.

If you're an existing instance provider using koios-1.0.8, the update process would be executing :

./setup-grest.sh -r -q -b koios-1.0.9
sudo systemctl restart cnode-haproxy

If you had recently updated your instance to use koios-1.0.9rc, the update process would be executing:

./setup-grest.sh -q -b koios-1.0.9

Bug Reports/Feature Requests

Everyone is invited to create future feature requests here to help us triage issues.

Support/Discussions Group

Please ensure to join the Koios Support/Discussions group.
There are bi-weekly meetings held for development activities on Koios gRest project at every 2nd and 4th Thursday of the month, the details for which can be found in the group above.