Configuration - how do you configure the reviews?

[sammcj]

Again - love this app/bot, it's great!

I'd love to be able to configure the reviews, I see on the apps landing page this is mentioned as a feature - but I can't see any docs on where to add a configuration file / set parameters - is this possible at present?

Example review: sammcj/gollama#140 (review)

• Currently the critical findings are at the end of the review and they're collapsed by default, it would be nice if these could be expanded as they're arguably the most important to read through and perhaps the other sections collapsed?
• After merging this PR, my contributors bot raised a new PR to add the contributor to my readme, it would be nice to disable LPR reviews on such PRs (e.g. docs(contributor): contributors readme action update sammcj/gollama#141)
• Also note the broken markdown code block on this review

[JetXu-LLM]

Hi @sammcj, thanks for your detailed feedback and questions! Let me address each point:

Review Configuration Status

Currently, the customizable review rules feature is still in development. While it's mentioned on our landing page as an upcoming feature, we apologize for any confusion this might have caused. We're actively working on making reviews more configurable and will announce when this feature becomes available.

Review Structure & Collapsible Sections

Our current review structure intentionally keeps three key sections expanded by default:

• Overview
• Detailed Technical Analysis (including code analysis and suggestions)
• Summary

Other sections are collapsed to maintain focus on the most critical information. However, we're interested in your suggestion about critical findings visibility. Would you prefer:

1. Critical findings section to be expanded by default?
2. Different sections to be collapsed/expanded?

Markdown Issues & Recent Improvements

Good news! We've recently (3 days ago) made significant improvements to address the markdown formatting issues:

• Upgraded from Mistral-large-2407 to Mistral-large-2411
• Optimized our prompt handling based on the latest best practices
• Enhanced content processing to prevent markdown breakage and PR review comment truncation

These changes should significantly improve the review comment quality. Please let us know if you notice any remaining issues!

PR Skip Rules

Regarding the contributors readme PR (sammcj/gollama#141), I'm glad to confirm that LlamaPReview already successfully identified and skipped it automatically. You should have seen a "Review Status: Automated Review Skipped" comment.

If you have suggestions for additional PR patterns that should be automatically skipped, we'd love to hear them to improve our skip rules further.

Looking forward to your thoughts on these points! 🚀

[sammcj]

Review Structure & Collapsible Sections

I think 'Critical findings section to be expanded by default' would be great - given that's arguably the most important information in the comment - and the comment can be quite big.

Markdown Issues & Recent Improvements

Nice work!

I hope you're getting some funding to cover the inference costs of it, it's a great model but pretty large.

PR Skip Rules

Oh sorry perhaps I wasn't clear, would it be possible to have it not comment at all on skipped PRs? Sometimes when using other bots or automations having unexpected comments added can mess them up.

[JetXu-LLM]

Thank you for your detailed feedback! Let me address each point:

Review Structure

Critical findings section to be expanded by default

We're currently running a community poll to determine which sections should be shown by default. Your perspective aligns with our thinking about the importance of critical findings. Please consider voting to help shape the final template structure!

Markdown & Costs

Thanks for the kind words! Yes, we're actively managing the inference costs while maintaining high-quality reviews 🚀

PR Skip Rules

Regarding the skip behavior - we've recently enhanced our skip logic with:

1. More sophisticated skip rules
2. LLM-based intelligent skip detection

We maintain a minimal default comment when skipping to indicate that LlamaPReview is working as intended rather than failing silently. This helps users distinguish between a successful skip and a potential system failure.