Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[awsfirehosereceiver] Add auto type #36708

Open
wants to merge 15 commits into
base: main
Choose a base branch
from

Conversation

constanca-m
Copy link

@constanca-m constanca-m commented Dec 6, 2024

Please see the issue #35988 (comment).

This new record_type does the exact same thing as the other record types, but is capable of distinguishing between them:

  • If it is JSON:
    • If it is a metrics receiver, parses it as cloudwatch metric.
    • If it is a logs receiver, parses it as a cloudwatch log.
  • If it is protobuf:
    • If it is a metric, parses it as OTLP metric.

I have used the exact same tests for this record type as the ones that are present for the other record types so there isn't any doubt that the new record type works.

@constanca-m
Copy link
Author

constanca-m commented Dec 6, 2024

Referencing #35988 (comment), @Aneurysm9 , does this approach look better? We check the bytes, we don't do multiple unmarshalls and we are capable now of not having to manually distinguish between these types.

Copy link
Contributor

This PR was marked stale due to lack of activity. It will be closed in 14 days.

@github-actions github-actions bot added the Stale label Dec 25, 2024
receiver/awsfirehosereceiver/receiver.go Outdated Show resolved Hide resolved
@@ -159,13 +158,6 @@ func TestFirehoseRequest(t *testing.T) {
wantStatusCode: http.StatusInternalServerError,
wantErr: firehoseConsumerErr,
},
"WithCorruptBase64Records": {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why remove this?

Copy link
Author

@constanca-m constanca-m Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, my first reply to this was wrong. Before this PR, if there was a wrong record, then everything else would fail. Now this does not happen anymore (see this line), since we have a continue statement instead of throwing an error.

I did this because we follow the same logic for all unmarshallers, and I wanted to keep it consistent here too.

I have added the test back again, but now with the OK status.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I was thinking of proposing that unmarshallers stop ignoring errors - I'm not aware that other receivers that behave like this, but maybe just out of ignorance. At any rate, I'd prefer that we don't expand this behaviour. If it's not strictly necessary for this PR, can we revert to the original behaviour, and keep it for another PR?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I have reverted it now.

@github-actions github-actions bot removed the Stale label Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants