Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for excluding functions from coverage with attribute #2593

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

benjaminfuchs
Copy link

@benjaminfuchs benjaminfuchs commented Dec 21, 2024

PR Summary

Fixes #2268

This PR adds support for excluding specific code constructs from coverage analysis using [ExcludeFromCodeCoverageAttribute()].

Changes:

  • New CoverageLocationVisitor: Skips AST nodes in script blocks marked with [ExcludeFromCodeCoverageAttribute()].
  • Get-CommandsInFile updated: Replaced predicate-based filtering with CoverageLocationVisitor.
  • Updated Tests: Ensures nodes with [ExcludeFromCodeCoverageAttribute()] are skipped.
    Verifies inclusion of unmarked nodes and relevant control flow statements.

PR Checklist

  • PR has meaningful title
  • Summary describes changes
  • PR is ready to be merged
    • If not, use the arrow next to Create Pull Request to mark it as a draft. PR can be marked Ready for review when it's ready.
  • Tests are added/update (if required)
  • Documentation is updated/added (if required)

@benjaminfuchs
Copy link
Author

If the approach is valid, I'll refine the tests and add proper documentation. Feedback is welcome!

@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch 3 times, most recently from 99eb66a to 1c9dd96 Compare December 21, 2024 22:37
@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch from 1c9dd96 to d1c84df Compare December 21, 2024 22:58
@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch from 53c13ac to 8608926 Compare December 29, 2024 12:48
Copy link
Collaborator

@fflaten fflaten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for looking into this! It works as intended, though I've suggested some improvements.

While reviewing I realized we could make this more efficient, see comment about FindAll. So I'd like to discuss an alternative approach. See draft PR #2598 which should produce the same results.

src/functions/Coverage.ps1 Outdated Show resolved Hide resolved
@@ -297,17 +297,101 @@ function Get-CommandsInFile {
$args[0] -is [System.Management.Automation.Language.BreakStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ContinueStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ExitStatementAst] -or
$args[0] -is [System.Management.Automation.Language.ThrowStatementAst]
$args[0] -is [System.Management.Automation.Language.ThrowStatementAst] -and
-not (IsExcludedByAttribute -Ast $args[0] -TargetAttribute 'System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FindAll will trigger this for commands in nested scriptblocks even though a parent is excluded. Not ideal to reverse (check parents) on every step.

$ast = {
    1 | ForEach-Object {
        [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute()]
        param($a, $b)
        $a + $b
        2 | ForEach-Object {
            3 | ForEach-Object {
                continue
             }
        }
    }
}.Ast
$ast.FindAll($predicate, $searchNestedScriptBlocks)

# Processing .. output added in IsExcludedByAttribute

Processing 1
Processing ForEach-Object {
        [System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute()]
        param($a, $b)
        $a + $b
        2 | ForEach-Object {
            3 | ForEach-Object {
                continue
             }
        }
    }
Processing $a + $b
Processing 2
Processing ForEach-Object {
            3 | ForEach-Object {
                continue
             }
        }
Processing 3
Processing ForEach-Object {
                continue
             }
Processing continue

Expression Redirections Extent
---------- ------------ ------                                                                                                                                      
1          {}           1                                                                                                                                           
           {}           ForEach-Object {… 

We could avoid this by rewriting to use an AstVisitor, stopping early at a ScriptBlockAst where the attribute is set.

src/functions/Coverage.ps1 Outdated Show resolved Hide resolved
src/functions/Coverage.ps1 Outdated Show resolved Hide resolved
src/functions/Coverage.ps1 Outdated Show resolved Hide resolved
src/functions/Coverage.ps1 Outdated Show resolved Hide resolved
@benjaminfuchs
Copy link
Author

@fflaten Thanks for the feedback! I agree with your point about improving efficiency by replacing FindAll with Visit. The approach in draft PR #2598 looks good to me. My check for the attribute also seems a bit over-engineered. If you'd like, I can take over your implementation and finalize the PR with tests and documentation updates, or let me know how you'd like to proceed.

@fflaten
Copy link
Collaborator

fflaten commented Jan 12, 2025

Looks like you've simplified yours as well. Feel free the use and complete the Visitor-based code. Was waiting on @nohwnd's thoughts but personally I'd prefer it as it's more extensible.

@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch 2 times, most recently from c1acbcd to e800b90 Compare January 12, 2025 21:26
@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch 2 times, most recently from 9865db2 to efd3c4c Compare January 12, 2025 23:29
@benjaminfuchs benjaminfuchs force-pushed the feature/exclude-attribute-coverage branch from efd3c4c to 82ef868 Compare January 12, 2025 23:32
@nohwnd
Copy link
Member

nohwnd commented Jan 13, 2025

Will try to have a look today in the evening.

$visitor.CoverageLocations.Count | Should -Be 7 -Because "Break, Continue, Throw, and Exit statements should be collected."
}
}
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be nice to have a more high-level api test here, similar to the "Single class using '" tests above. Where we enter and leave the coverage and get the results.


namespace Pester
{
public class CoverageLocationVisitor : AstVisitor2
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some short info about what this class does if possible. The original predicate is somehow (at least for me) easier to grasp what the intent is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And possibly also why we prefer it over the original predicate.

@nohwnd
Copy link
Member

nohwnd commented Jan 13, 2025

Great job here,thank you! Some small nitpicks on the code :)

@fflaten
Copy link
Collaborator

fflaten commented Jan 14, 2025

Thanks. @benjaminfuchs Let me know if you'd like me to address any of the comments.

@nohwnd
Copy link
Member

nohwnd commented Jan 14, 2025 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow [ExcludeFromCodeCoverage()] on the param() block to exclude a function from code coverage
3 participants