Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

testing new db approach on CI #402

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open

testing new db approach on CI #402

wants to merge 3 commits into from

Conversation

arkanoider
Copy link
Collaborator

@arkanoider arkanoider commented Dec 15, 2024

Hi @grunch , @Catrya , @bilthon

being in office today and so a bit more in trouble to work on mostro testing, i tweaked a bit for the mostro.db "issue".
I mean... we also been there...time passes me or @grunch change something in sql files and the you pull latest work try to create an order...take it...and boom db error!! Why?! WTF! It's always the same ( @Catrya knows well 😄 ).

So you need to run init_db.sh and copy your new db in mostro folder.

I was trying to use build scritp (build.rs) to improve this and automatically generate at every build the correct db, but murphy's law hit me and I run into this so called deadlock issue:

https://lpalmieri.com/posts/cargo-px/#invoking-cargo

and i admit I was not able to exit from there easily.

Ok! After this what i am proposing here is to generate:

  • mostro.db
  • sqlx-data.json

from github actions. Not was super immediate, but I think it's working now. Some details:

  • I put both files in .gitignore so we can play locally and if we modify sql file correct files will be autogenerated from github and committed to repo.
  • Result of this is that when you pull latest modification you should have both files updated, eventually the missing point is just the copy of mostro.db in mostro folder, means no more init_db.sh for normal user at least he wants to change locally sql files in migrations folder.
  • So in the end the new .yml file in mostro will do all the job that init_db did locally but in the github world creating the file for us.

The downside is a commit more generated by the actions, not a big issue imo, but i am here to listen your opinion.

Take a look!

Summary by CodeRabbit

  • New Features

    • Enhanced CI workflow for Rust with integrated database setup and management.
    • Added environment variable DATABASE_URL for database configuration.
  • Bug Fixes

    • Adjusted .gitignore to track variations of mostro.db while ignoring the specific file.
    • Added entry for sqlx-data.json in .gitignore.
  • Chores

    • Updated permissions for workflow to allow commit and push actions.

Copy link
Contributor

coderabbitai bot commented Dec 15, 2024

Warning

Rate limit exceeded

@arkanoider has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 1 minutes and 28 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

📥 Commits

Reviewing files that changed from the base of the PR and between f41dd3b and dc4b295.

⛔ Files ignored due to path filters (1)
  • mostro.db is excluded by !**/*.db
📒 Files selected for processing (2)
  • .github/workflows/rust.yml (1 hunks)
  • .gitignore (1 hunks)

Walkthrough

The pull request introduces modifications to the GitHub Actions workflow for Rust and the .gitignore file. The workflow configuration now includes enhanced database management steps, with new environment variables and permissions. The workflow adds steps to install sqlx-cli, create and manage a SQLite database, and commit generated database files back to the repository. The .gitignore file has been updated to specifically ignore mostro.db and sqlx-data.json files.

Changes

File Change Summary
.github/workflows/rust.yml - Added DATABASE_URL environment variable
- Granted GITHUB_TOKEN write permissions
- Updated checkout steps for pull requests and main branch
- Added steps for database management and file generation
.gitignore - Changed mostro.db* to mostro.db
- Added sqlx-data.json to ignored files

Poem

🐰 A Rabbit's CI Dance 🔧

In workflows of Rust, a new beat thrums,
Database migrations, where code becomes
A symphony of commits and care,
With sqlx-cli beyond compare!

Hop, hop, through GitHub's green lane! 🚀


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@arkanoider arkanoider force-pushed the ci-improve-db branch 2 times, most recently from 9c15dff to 6beea3d Compare December 16, 2024 16:25
@arkanoider arkanoider marked this pull request as ready for review December 16, 2024 16:49
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (3)
.github/workflows/rust.yml (3)

14-14: Consider using absolute path for DATABASE_URL

While the SQLite URL format is correct, using a relative path (./mostro.db) in CI environment could be fragile. Consider using an absolute path to ensure consistent behavior.

-  DATABASE_URL: sqlite://./mostro.db
+  DATABASE_URL: sqlite://${{ github.workspace }}/mostro.db

30-40: Optimize branch selection logic

The current implementation has duplicate checkout steps that could be simplified into a single step using a dynamic reference.

-      - name: Select branch PR
-        if: ${{ github.ref != 'refs/heads/main' }}
-        uses: actions/checkout@v4        
-        with:
-          ref: ${{ github.event.pull_request.head.ref }}
-        
-      - name: Select branch main
-        if: ${{ github.ref == 'refs/heads/main' }}
-        uses: actions/checkout@v4        
-        with:
-          ref: main
+      - name: Checkout code
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || 'main' }}
🧰 Tools
🪛 yamllint (1.35.1)

[error] 32-32: trailing spaces

(trailing-spaces)


[error] 35-35: trailing spaces

(trailing-spaces)


[error] 38-38: trailing spaces

(trailing-spaces)


24-24: Fix YAML formatting issues

Remove trailing spaces from lines 24, 25, 29, 32, 35, and 38 to maintain consistent formatting.

Also applies to: 25-25, 29-29, 32-32, 35-35, 38-38

🧰 Tools
🪛 yamllint (1.35.1)

[error] 24-24: trailing spaces

(trailing-spaces)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3e5f7fa and 5a40afc.

⛔ Files ignored due to path filters (1)
  • mostro.db is excluded by !**/*.db
📒 Files selected for processing (2)
  • .github/workflows/rust.yml (1 hunks)
  • .gitignore (1 hunks)
✅ Files skipped from review due to trivial changes (1)
  • .gitignore
🧰 Additional context used
🪛 yamllint (1.35.1)
.github/workflows/rust.yml

[error] 24-24: trailing spaces

(trailing-spaces)


[error] 25-25: trailing spaces

(trailing-spaces)


[error] 29-29: trailing spaces

(trailing-spaces)


[error] 32-32: trailing spaces

(trailing-spaces)


[error] 35-35: trailing spaces

(trailing-spaces)


[error] 38-38: trailing spaces

(trailing-spaces)

🔇 Additional comments (1)
.github/workflows/rust.yml (1)

20-23: Review security implications of broad write permissions

While write permissions are needed for committing database files, granting broad contents: write access might be more permissive than necessary. Consider if a more restricted scope would suffice.

✅ Verification successful

Write permissions are appropriately scoped for the workflow's needs

The contents: write permission is justified here as the workflow specifically needs to commit two files to the repository:

  • mostro.db (SQLite database file)
  • sqlx-data.json (SQLx offline mode data)

These commits are done through the EndBug/add-and-commit action with explicit file paths, which provides a controlled way of modifying repository contents. The broad write permission is necessary for this operation but the actual usage is well-constrained by the action's configuration.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Check if the workflow modifies files outside of expected paths
gh api graphql -f query='
query($owner:String!, $name:String!, $branch:String!) {
  repository(owner:$owner, name:$name) {
    object(expression:$branch) {
      ... on Commit {
        history(first:10) {
          nodes {
            author {
              name
            }
            messageHeadline
            additions {
              path
            }
          }
        }
      }
    }
  }
}' -F owner='MostroP2P' -F name='mostro' -F branch='main' | \
jq '.data.repository.object.history.nodes[] | select(.author.name=="GitHub Actions")'

Length of output: 702


Script:

#!/bin/bash
# Let's check the workflow file to understand what files it actually modifies
cat .github/workflows/rust.yml

# Also check for any other workflow files that might provide context
fd -e yml -e yaml . .github/workflows/

Length of output: 1749

.github/workflows/rust.yml Outdated Show resolved Hide resolved
.github/workflows/rust.yml Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
.github/workflows/rust.yml (1)

60-69: ⚠️ Potential issue

Remove force flag from file additions

Using the --force flag could potentially cause issues with local development.

Apply this change to prevent potential conflicts:

           add: |
-            - mostro.db --force
-            - sqlx-data.json --force
+            - mostro.db
+            - sqlx-data.json
🧹 Nitpick comments (3)
.github/workflows/rust.yml (3)

30-40: Simplify branch selection logic

The current implementation uses two separate checkout steps, which adds unnecessary complexity.

Consider simplifying to a single checkout step:

-      - name: Select branch PR
-        if: ${{ github.ref != 'refs/heads/main' }}
-        uses: actions/checkout@v4        
-        with:
-          ref: ${{ github.event.pull_request.head.ref }}
-        
-      - name: Select branch main
-        if: ${{ github.ref == 'refs/heads/main' }}
-        uses: actions/checkout@v4        
-        with:
-          ref: main
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          ref: ${{ github.event_name == 'pull_request' && github.event.pull_request.head.ref || 'main' }}
🧰 Tools
🪛 yamllint (1.35.1)

[error] 32-32: trailing spaces

(trailing-spaces)


[error] 35-35: trailing spaces

(trailing-spaces)


[error] 38-38: trailing spaces

(trailing-spaces)


42-43: Consider using version range for sqlx-cli

While version pinning provides stability, it might prevent getting important updates and security fixes.

Consider using a version range:

-        run: cargo install sqlx-cli --version 0.6.2
+        run: cargo install sqlx-cli --version "^0.6.2"

60-69: Consider optimizing commit step to avoid unnecessary commits

The current implementation will create commits even when there are no changes to the database files.

Consider adding a check for changes before committing:

       - name: Commit mostro.db and sqlx-data.json
+        run: |
+          if [[ -n "$(git status --porcelain mostro.db sqlx-data.json)" ]]; then
         uses: EndBug/add-and-commit@v9
         with:
           default_author: github_actions
           message: 'chore(db): Update mostro.db and sqlx-data.json [skip ci]'
           committer_name: GitHub Actions
           committer_email: [email protected]
           add: |
             - mostro.db
             - sqlx-data.json
+        fi
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 5a40afc and f41dd3b.

📒 Files selected for processing (1)
  • .github/workflows/rust.yml (1 hunks)
🧰 Additional context used
🪛 yamllint (1.35.1)
.github/workflows/rust.yml

[error] 24-24: trailing spaces

(trailing-spaces)


[error] 25-25: trailing spaces

(trailing-spaces)


[error] 29-29: trailing spaces

(trailing-spaces)


[error] 32-32: trailing spaces

(trailing-spaces)


[error] 35-35: trailing spaces

(trailing-spaces)


[error] 38-38: trailing spaces

(trailing-spaces)


[error] 45-45: trailing spaces

(trailing-spaces)


[error] 46-46: trailing spaces

(trailing-spaces)


[error] 47-47: trailing spaces

(trailing-spaces)


[error] 51-51: trailing spaces

(trailing-spaces)


[error] 52-52: trailing spaces

(trailing-spaces)


[error] 53-53: trailing spaces

(trailing-spaces)

🔇 Additional comments (2)
.github/workflows/rust.yml (2)

14-14: LGTM: Environment and permissions are properly configured

The DATABASE_URL and write permissions are correctly set up to support the automated database management workflow.

Also applies to: 20-23


55-58: LGTM: SQLx preparation steps are well-implemented

The preparation and verification of the offline file are properly configured to support offline compilation.

@grunch
Copy link
Member

grunch commented Dec 23, 2024

If I understand this correctly, with every new commit a new commit on mostrod repository will be generated automatically?

@arkanoider
Copy link
Collaborator Author

If I understand this correctly, with every new commit a new commit on mostrod repository will be generated automatically?

You're right!

Was not able to do it better,but now:

  • We have always the latest mostro.db ( and also local json file) generated for users ( devs will find another commit after their latest and this could be a downside )
  • We just need eventually to remind users to overwrite their mostro.db file without using init_db

Don't know if you like it, I see some advanteges, also if not a perfect solution.
Still have to see the behaviour if you pull from repo and you have a mostro.db that must be updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants