[receiver/filelog] Implement ExcludeOlderThan matcher criterion
#31916
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ycombinator
force-pushed
the
receiver-filelog-exclude-older-than
branch
from
6d99e99 to
57819c0
Compare
ChrsMark
reviewed
Mar 28, 2024
djaglowski
reviewed
Mar 28, 2024
ycombinator
force-pushed
the
receiver-filelog-exclude-older-than
branch
from
7ee2a2e to
e802024
Compare
6 tasks
|
Please address CI failures |
ycombinator
requested review from
MovieStoreGuy,
andrzej-stencel,
crobert-1,
dashpole,
atoulme,
jmacd,
dmitryax,
codeboten,
fatsheep9146,
jpkrohling,
Aneurysm9,
TylerHelmuth,
yurishkuro,
mx-psi,
bogdandrutu and
evan-bradley
as code owners
|
Ugh, I had a bad merge which resulted in a lot of files being changed and lots of labels and reviewers being added to this PR. I've sinced fixed the commits but the labels and reviewers remain. Sorry about that. 😞 @djaglowski should I close this PR and open a new one or do you want to remove the unrelated labels and reviewers from this PR? |
bryan-aguilar
removed
exporter/alibabacloudlogservice
|
I have a use case which is similar to this where i want to do
If your change gets merged, and we set both exclude_older_than: 1h
ordering_criteria:
top_n: 3
sort_by:
- sort_type: mtimewill it work as described above? |
djaglowski
approved these changes
Apr 15, 2024
Comment on lines
+45
to
+46
| fileMTimes: []time.Time{twoHoursAgo, threeHoursAgo}, | ||
| excludeOlderThan: 3 * time.Hour, |
There was a problem hiding this comment.
This can be a followup PR but I think the test is set up a bit strange here because we're comparing "exactly 3 hours" to "3 hour + trivial time" (because we initialize threeHoursAgo above and compare it just a few cycles later.) I think we should instead make this an unambiguous comparison and consider ways to test the exactly equal case.
ghost
pushed a commit
to opsramp/opentelemetry-collector-contrib
that referenced
this pull request
May 5, 2024
…en-telemetry#31916) **Description:** This PR implements a new matcher criterion in the Stanza fileconsumer matcher: ``` ExcludeOlderThan time.Duration `mapstructure:"exclude_older_than"` ``` and the corresponding setting in the `filelog` receiver configuration: | Field | Default | Description | |-------------------------------------|--------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | `exclude_older_than` | | Exclude files whose modification time is older than the specified age. | When specified, the matcher will exclude files whose modification times are older than the specified time. **Link to tracking Issue:** open-telemetry#31053 **Testing:** Added unit tests. **Documentation:** Documented `exclude_older_than` configuration setting in the `filelogreceiver`'s README. --------- Co-authored-by: Daniel Jaglowski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description:
This PR implements a new matcher criterion in the Stanza fileconsumer matcher:
and the corresponding setting in the
filelogreceiver configuration:exclude_older_thanWhen specified, the matcher will exclude files whose modification times are older than the specified time.
Link to tracking Issue: #31053
Testing: Added unit tests.
Documentation: Documented
exclude_older_thanconfiguration setting in thefilelogreceiver's README.