Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add feature to collect function/statement/branch coverage to test command #343

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open

Add feature to collect function/statement/branch coverage to test command #343

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
7ae1237
Add -c / --coverage option to switch coverage measurement
nodaguti Jul 21, 2024
5a3b16b
Instrument code to measure coverage
nodaguti Jul 21, 2024
ed62b6e
Calculate coverage after running all tests
nodaguti Jul 21, 2024
1d062a7
Add tests
nodaguti Jul 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion cmd/falco/help.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ func printSplash() {
writeln(white, strings.TrimSpace(`
=========================================================
____ __
/ __/______ / /_____ ____
/ __/______ / /_____ ____
/ /_ / __ `+` // // __// __ \
/ __// /_/ // // /__ / /_/ /
/_/ \____//_/ \___/ \____/ Fastly VCL developer tool
Expand Down Expand Up @@ -145,6 +145,7 @@ Flags:
-r, --remote : Connect with Fastly API
-t, --timeout : Set timeout to running test
-f, --filter : Override glob filter to find test files
-c, --coverage : Collect test coverage information and report it in the output
-json : Output results as JSON
-request : Override request config
--max_backends : Override max backends limitation
Expand Down
17 changes: 13 additions & 4 deletions cmd/falco/main.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -289,11 +289,13 @@ func runTest(runner *Runner, rslv resolver.Resolver) error {
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
if err := enc.Encode(struct {
Tests []*tester.TestResult `json:"tests"`
Summary *tester.TestCounter `json:"summary"`
Tests []*tester.TestResult `json:"tests"`
Summary *tester.TestCounter `json:"summary"`
Coverage *tester.TestCoverage `json:"coverage,omitempty"`
}{
Tests: factory.Results,
Summary: factory.Statistics,
Tests: factory.Results,
Summary: factory.Statistics,
Coverage: factory.Coverage,
}); err != nil {
writeln(red, err.Error())
return ErrExit
Expand Down Expand Up @@ -379,6 +381,13 @@ func runTest(runner *Runner, rslv resolver.Resolver) error {
write(white, "%d total, ", totalCount)
writeln(white, "%d assertions", factory.Statistics.Asserts)

if factory.Coverage != nil {
write(white, "Coverage: ")
write(white, "%% Stmts: %.2f, ", factory.Coverage.Statement*100)
write(white, "%% Branch: %.2f, ", factory.Coverage.Branch*100)
writeln(white, "%% Funcs: %.2f", factory.Coverage.Function*100)
}

if factory.Statistics.Fails > 0 {
return ErrExit
}
Expand Down
1 change: 1 addition & 0 deletions config/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ type SimulatorConfig struct {
type TestConfig struct {
Timeout int `cli:"t,timeout" yaml:"timeout"`
Filter string `cli:"f,filter" default:"*.test.vcl"`
Coverage bool `cli:"c,coverage" yaml:"coverage" default:"false"`
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the our circumstance, could the coverage option be a --converage - only long option?
This is because we'd like to use -c short option for the other usage like config path definition.

IncludePaths []string // Copy from root field
OverrideHost string `yaml:"host"`

Expand Down
3 changes: 2 additions & 1 deletion docs/configuration.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Here is a full configuration file example:
// .falco.yaml

## Basic configurations
include_paths: [".", "/path/to/include"]
include_paths: [".", "/path/to/include"]
remote: true
max_backends: 5
max_acls: 1000
Expand Down Expand Up @@ -65,6 +65,7 @@ All configurations of configuration files and CLI arguments are described follow
| simulator.edge_dictionary.[name] | Object | - | - | Local edge dictionary name |
| testing | Object | null | - | Testing configuration object |
| testing.timeout | Integer | 10 | -t, --timeout | Set timeout to stop testing |
| testing.coverage | Boolean | false | -c, --coverage | Collect test coverage information and report it in the output |
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the our circumstance, could the coverage option be a --converage - only long option?
This is because we'd like to use -c short option for the other usage like config path definition.

| linter | Object | null | - | Override linter rules |
| linter.verbose | String | error | -v, -vv | Verbose level, `warning` or `info` is valid |
| linter.rules | Object | null | - | Override linter rules |
Expand Down
Loading