Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[OSS-ONLY] Implementing hashcode and mode based caching #3414

Open
wants to merge 23 commits into
base: BABEL_5_X_DEV
Choose a base branch
from

Conversation

SiddharthBITS
Copy link

@SiddharthBITS SiddharthBITS commented Jan 16, 2025

Description
This commit implements a more efficient method of caching compilations. Cache now restored will now be guaranteed to compatible with the version and build configuration of compilations.

We will now save 5-8 mins in version upgrade and dump restore tests.

Key changes:

  • Implemented new format for cache keys which includes github hashcode of corresponding engine branch and build mode to better identify the cache stored in a key.
  • New format "ccache-engineHashcode-buildMode-extensionHashcode".
  • Cache restoration handled by build-modified-postgres.yml and saving by save-ccache.yml composite actions.
  • Removed timeStamp based caching.

Task: BABEL-5564
Signed-off-by: Siddharth Sengar [email protected]

Check List

  • Commits are signed per the DCO using --signoff

By submitting this pull request,
I confirm that my contribution is under the terms of the Apache 2.0 and PostgreSQL licenses, and grant any person obtaining a copy of the contribution permission to relicense all or a portion of my contribution to the PostgreSQL License solely to contribute all or a portion of my contribution to the PostgreSQL open source project.

For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@coveralls
Copy link
Collaborator

coveralls commented Jan 16, 2025

Pull Request Test Coverage Report for Build 12830872482

Warning: This coverage report may be inaccurate.

This pull request's base commit is no longer the HEAD commit of its target branch. This means it includes changes from outside the original pull request, including, potentially, unrelated coverage changes.

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.003%) to 74.956%

Totals Coverage Status
Change from base Build 12805401204: 0.003%
Covered Lines: 46993
Relevant Lines: 62694

💛 - Coveralls

@@ -24,11 +24,11 @@ inputs:
runs:
using: "composite"
steps:
- name: Checkout, Build, and Install the Modified PostgreSQL Instance and Run Tests
- name: Remake engine repository
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us name this checkout Modified PostgreSQL for Babelfish

rm -rf ~/.ccache
echo "CCACHE_KEY=" >> $GITHUB_ENV
echo "SAVE_CCACHE=" >> $GITHUB_ENV
echo "SAVE_CACHE_HERE=" >> $GITHUB_ENV
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us also unset CRESTORE_KEY

@SiddharthBITS SiddharthBITS changed the title [DO NOT MERGE] Implementing caching on 5X Implementing hashcode and mode based caching Jan 17, 2025
key:
random-random
restore-keys:
ccache-${{ env.CRESTORE_KEY }}
Copy link
Contributor

@shah-nirmit shah-nirmit Jan 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question: So this will only match by Engine SHA followed by whatever sha matches for extension, so if Extension has some new code changes will the action ensure the newest version of cache for that PR is used?

Copy link
Author

@SiddharthBITS SiddharthBITS Jan 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if [[ ${{ github.event_name != 'pull_request' || env.SAVE_CCACHE == 1 }} == true ]]; then
echo "Cache to be Saved"
echo "SAVE_CACHE_HERE=1" >> $GITHUB_ENV

We are force saving cache for any request that isn't a PR, so a merge would trigger this.

@SiddharthBITS SiddharthBITS changed the title Implementing hashcode and mode based caching [OSS-ONLY] Implementing hashcode and mode based caching Jan 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants