-
Notifications
You must be signed in to change notification settings - Fork 26
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: improve slice usage from pooled objects #160
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
2bb345d
feat: improve slice usage from pooled objects
kruskall fba9b12
lint: fix linter issues
kruskall 9b4b287
feat: add reslice to http headers
kruskall 8aded2e
feat: add reslice to metricset samples
kruskall 2e41c65
feat: add reslice to stacktrace frame
kruskall dcd60f2
feat: add reslice to cause
kruskall 038a9e9
feat: add reslice to spanlinks
kruskall bf46123
fix: stacktrace slice not being populated leading to nil pointer
kruskall 0fab4ff
docs: add reslice documentation
kruskall File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -22,20 +22,21 @@ import ( | |
"google.golang.org/protobuf/types/known/structpb" | ||
) | ||
|
||
func ToKv(m map[string]any) []*modelpb.KeyValue { | ||
nm := normalizeMap(m) | ||
if len(nm) == 0 { | ||
func ToKv(m map[string]any, out []*modelpb.KeyValue) []*modelpb.KeyValue { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Same here. |
||
m = normalizeMap(m) | ||
if len(m) == 0 { | ||
return nil | ||
} | ||
|
||
kv := []*modelpb.KeyValue{} | ||
out = Reslice(out, len(m), modelpb.KeyValueFromVTPool) | ||
|
||
i := 0 | ||
for k, v := range m { | ||
value, _ := structpb.NewValue(v) | ||
pbkv := modelpb.KeyValueFromVTPool() | ||
pbkv.Key = k | ||
pbkv.Value = value | ||
kv = append(kv, pbkv) | ||
out[i].Key = k | ||
out[i].Value = value | ||
i++ | ||
} | ||
|
||
return kv | ||
return out | ||
} |
34 changes: 34 additions & 0 deletions
34
input/elasticapm/internal/modeldecoder/modeldecoderutil/slice.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,34 @@ | ||
// Licensed to Elasticsearch B.V. under one or more contributor | ||
// license agreements. See the NOTICE file distributed with | ||
// this work for additional information regarding copyright | ||
// ownership. Elasticsearch B.V. licenses this file to you under | ||
// the Apache License, Version 2.0 (the "License"); you may | ||
// not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, | ||
// software distributed under the License is distributed on an | ||
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
// KIND, either express or implied. See the License for the | ||
// specific language governing permissions and limitations | ||
// under the License. | ||
|
||
package modeldecoderutil | ||
|
||
// Reslice increases the slice's capacity, if necessary, to match n. | ||
// If specified, the newFn function is used to create the elements | ||
// to populate the additional space appended to the slice. | ||
func Reslice[Slice ~[]model, model any](slice Slice, n int, newFn func() model) Slice { | ||
if diff := n - cap(slice); diff > 0 { | ||
extra := make([]model, diff) | ||
if newFn != nil { | ||
for i := range extra { | ||
extra[i] = newFn() | ||
} | ||
} | ||
slice = append([]model(slice)[:cap(slice)], extra...) | ||
} | ||
return slice[:n] | ||
} |
50 changes: 50 additions & 0 deletions
50
input/elasticapm/internal/modeldecoder/modeldecoderutil/slice_test.go
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
// Licensed to Elasticsearch B.V. under one or more contributor | ||
// license agreements. See the NOTICE file distributed with | ||
// this work for additional information regarding copyright | ||
// ownership. Elasticsearch B.V. licenses this file to you under | ||
// the Apache License, Version 2.0 (the "License"); you may | ||
// not use this file except in compliance with the License. | ||
// You may obtain a copy of the License at | ||
// | ||
// http://www.apache.org/licenses/LICENSE-2.0 | ||
// | ||
// Unless required by applicable law or agreed to in writing, | ||
// software distributed under the License is distributed on an | ||
// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | ||
// KIND, either express or implied. See the License for the | ||
// specific language governing permissions and limitations | ||
// under the License. | ||
|
||
package modeldecoderutil | ||
|
||
import ( | ||
"testing" | ||
|
||
"github.com/elastic/apm-data/model/modelpb" | ||
"github.com/stretchr/testify/assert" | ||
) | ||
|
||
func TestReslice(t *testing.T) { | ||
originalSize := 10 | ||
s := make([]*modelpb.APMEvent, originalSize) | ||
for i := range s { | ||
s[i] = &modelpb.APMEvent{} | ||
} | ||
|
||
downsize := 4 | ||
s = Reslice(s, downsize, nil) | ||
for _, e := range s { | ||
assert.NotNil(t, e) | ||
} | ||
assert.Equal(t, downsize, len(s)) | ||
assert.Equal(t, originalSize, cap(s)) | ||
|
||
upsize := 20 | ||
s = Reslice(s, upsize, func() *modelpb.APMEvent { return &modelpb.APMEvent{} }) | ||
assert.Equal(t, upsize, len(s)) | ||
for _, e := range s { | ||
assert.NotNil(t, e) | ||
} | ||
assert.Equal(t, upsize, cap(s)) | ||
|
||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This doesn't seem right, we're passing out and also returning out?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If
out
is not big enough we might append to it, so we need to return a slice because they might not matchThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find this API a bit confusing, I think it'd be best to do one or the other. If we want to accept the
out
as an argument, then we can passout *[]*modelpb.HTTPHeader
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess this signature looks similar to
s = append(s, struct{})
, so that's an argument for this pattern. But we'll have to remember to use the return value.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
But for conversion purposes like this specific one, I think @marclop 's comment makes sense. It will be clearer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm a bit conflicted. This method was inspired by the
slices
package in the stdlib (https://pkg.go.dev/slices)There are several methods/API there that returns the result (modified slice) despite changing the argument (Compact, Grow, Replace, Clip, etc.). I assume they might in turn have copied the signature of
append
.I think this is a common pattern in go ecosystem but I don't feel strongly about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for chiming in. I was unaware of the widespread apdoption of this pattern in the current Go stdlib. I still think it's clearer and simpler to accept a single argument. If we leave it as is, I would prefer if we documented the behaviour since there are currently no godocs for these functions.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done!