Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add id column as PRIMARY KEY for evm.logs & evm.log_poller_blocks #1441

Open
wants to merge 13 commits into
base: ccip-develop
Choose a base branch
from
Open

Add id column as PRIMARY KEY for evm.logs & evm.log_poller_blocks #1441

wants to merge 13 commits into from
+393 −281

Commits on Sep 20, 2024

  1. Manually merged in from chainlink repo: 

      BCI-3492 [LogPoller]: Allow withObservedExecAndRowsAffected to report non-zero rows affected (#14057)

    * Fix withObservedExecAndRowsAffected

    Also:
    - Change behavior of DeleteExpiredLogs to delete logs which don't match any filter
    - Add a test case to ensure the dataset size is published properly during pruning

    * pnpm changeset

    * changeset #fix -> #bugfix
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    3cc98be View commit details
    Browse the repository at this point in the history

  2. Add id column as PRIMARY KEY for evm.logs & evm.log_poller_blocks 

    Also:
   - Add UNIQUE INDEXes to replace previous primary keys (still necessary, both for
     optimizing queries and for enforcing uniqueness constraints)
   - Replace all SELECT *'s with helper functions for selecting all columns
   - Refactor nestedBlockQuery into withConfs, and make a bit more use of it
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    ed144e0 View commit details
    Browse the repository at this point in the history

  3. Clean up db indexes 

    Some of the columns in these indexes (such as created_at) are no longer used.
Others were not optimized for the queries we need.
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    9ace2c0 View commit details
    Browse the repository at this point in the history

  4. Fix 2 unrelated bugs I noticed

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    90dd2e7 View commit details
    Browse the repository at this point in the history

  5. Update ExpiredLogs query

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    ac60768 View commit details
    Browse the repository at this point in the history

  6. Update test for fromBlock >= :block_number 

    Previously it was using fromBlock > :block_number which is inconsistent
with the other fromBlocks in queries
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    bb4d81b View commit details
    Browse the repository at this point in the history

  7. Increase staggering of initial pruning runs

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    e8268f0 View commit details
    Browse the repository at this point in the history

  8. Decrease retention periods for CCIP events, for testing

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    e1fc86b View commit details
    Browse the repository at this point in the history

  9. Fix bug in merged commit from develop 

    On a node with more than one chain, each LogPoller would have deleted all logs from chains it's not
running on! Because of the LEFT JOIN, ON evm_chain_id = $1 does not filter out any rows where evm_chain_id != $1;
only WHERE evm_chain_id = $1 can do that
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    49e2fa7 View commit details
    Browse the repository at this point in the history

  10. restore retention periods

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    5bfea8c View commit details
    Browse the repository at this point in the history

  11. Set LogPrunePageSize = 2001

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    7a0568d View commit details
    Browse the repository at this point in the history

  12. Update DeleteBlocksBefore query to use block_number index instead of … 

    …LIMIT
    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    1ff9472 View commit details
    Browse the repository at this point in the history

  13. Split off SelectUnmatchedLogs from DeleteExpiredLogs

    @reductionista
    reductionista committed Sep 20, 2024
    Configuration menu
    Copy the full SHA
    a99e37f View commit details
    Browse the repository at this point in the history