Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Log error when statsd server fails to start and fix airflow docs #36477

Merged
merged 3 commits into from
Sep 13, 2023

Conversation

belimawr
Copy link
Contributor

@belimawr belimawr commented Aug 31, 2023

Proposed commit message

This commit adds a log error when the statsd server fails to start, it
also improves the Airflow module documentation to make extra clear
that the module will start a statsd server.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
    - [ ] I have made corresponding changes to the documentation
    - [ ] I have made corresponding change to the default configuration files
    - [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

## Author's Checklist

How to test this PR locally

  1. Create the following metricbeat.yml:
    metricbeat.modules:
    - module: airflow
      host: "1.2.3.4" # Any IP that does not belong to the host running Metricbeat
      port: "9102"
      ttl: "10s"
      period: 5s
      metricsets: [ 'statsd' ]
    
    output.console:
      enabled: true
  2. Start Metricbeat
  3. After a few seconds you should see a log error like this:
    {"log.level":"error","@timestamp":"2023-08-31T17:09:34.565+0200","log.logger":"airflow.statsd","log.origin":{"file.name":"server/server.go","file.line":201},"message":"could not start statsd server for module: airflow","service.name":"metricbeat","error":{"message":"failed to start UDP server: listen udp 1.2.3.4:9102: bind: cannot assign requested address"},"ecs.version":"1.6.0"}

## Related issues
## Use cases
## Screenshots
## Logs

@belimawr belimawr requested a review from a team as a code owner August 31, 2023 15:19
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2023
@mergify
Copy link
Contributor

mergify bot commented Aug 31, 2023

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @belimawr? 🙏.
For such, you'll need to label your PR with:

  • The upcoming major version of the Elastic Stack
  • The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v8./d.0 is the label to automatically backport to the 8./d branch. /d is the digit

@belimawr belimawr added bug Team:Elastic-Agent Label for the Agent team labels Aug 31, 2023
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Aug 31, 2023
@belimawr belimawr added the backport-v8.10.0 Automated backport with mergify label Aug 31, 2023
@belimawr belimawr changed the title Add log error when statsd server fails start Add log error when statsd server fails start and fix airflow docs Aug 31, 2023
@elasticmachine
Copy link
Collaborator

elasticmachine commented Aug 31, 2023

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-09-12T06:31:58.485+0000

  • Duration: 72 min 38 sec

Test stats 🧪

Test Results
Failed 0
Passed 4665
Skipped 1014
Total 5679

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@shmsr shmsr changed the title Add log error when statsd server fails start and fix airflow docs Add log error when statsd server fails to start and fix airflow docs Sep 4, 2023
@shmsr shmsr changed the title Add log error when statsd server fails to start and fix airflow docs Log error when statsd server fails to start and fix airflow docs Sep 4, 2023
@shmsr
Copy link
Member

shmsr commented Sep 4, 2023

Looks good to me but I could see more adjustments that could be made to Airflow docs. Sample patch:

 This module collects metrics from
-https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html[Airflow metrics]. It runs a
-statsd server where airflow will send metrics to. The default metricset is `statsd`.
+https://airflow.apache.org/docs/apache-airflow/stable/logging-monitoring/metrics.html[Airflow metrics]. It runs a statsd server where airflow will send metrics to. The default metricset is `statsd`.
 
 [float]
 === Compatibility
 
-The Airflow module is tested with Airflow 2.1.0. It should work with version
-2.0.0 and later.
+The Airflow module is tested with Airflow 2.1.0. It should work with version 2.0.0 and later.
 
 [float]
 === Usage
-The Airflow module requires <<metricbeat-module-statsd,Statsd>> to receive Statsd metrics. Refer to the link for instructions about how to use Statsd.
+The Airflow module requires <<metricbeat-module-statsd,Statsd>> to receive statsd metrics. Refer to the link for instructions about how to use statsd.
 
-Add the following lines to your Airflow configuration file e.g. `airflow.cfg` ensuring `statsd_prefix` is left empty and replace `%METRICBEAT_HOST%` with the address metricbeat is running:
+Add the following lines to your Airflow configuration file e.g. `airflow.cfg` ensuring `statsd_prefix` is left empty and replace `%METRICBEAT_HOST%` with the address where metricbeat is running:

Copy link
Member

@shmsr shmsr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Left one comment. Also, CI is failing. Could you please also run make update?

This commit adds a log error when the statsd server fails to start, it
also improves the Airflow module documentation to make extra clear
that the module will start a statsd server.
@belimawr
Copy link
Contributor Author

@shmsr I added all your suggestions on e8dc1d6

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: rename the method receiver from b to m: s/b/m

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

not important though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a bigger change on the this file, I rather make it into a separated PR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The followup PR: #36591

I think when I first looked I was thinking about the opposite change 🤦‍♂️. It's a super small PR :D

@belimawr belimawr merged commit 9dcd8be into elastic:main Sep 13, 2023
6 of 8 checks passed
mergify bot pushed a commit that referenced this pull request Sep 13, 2023
)

This commit adds a log error when the statsd server fails to start, it
also improves the Airflow module documentation to make extra clear
that the module will start a statsd server.

Co-authored-by: subham sarkar <[email protected]>
(cherry picked from commit 9dcd8be)
Scholar-Li pushed a commit to Scholar-Li/beats that referenced this pull request Feb 5, 2024
…stic#36477)

This commit adds a log error when the statsd server fails to start, it
also improves the Airflow module documentation to make extra clear
that the module will start a statsd server.

Co-authored-by: subham sarkar <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-v8.10.0 Automated backport with mergify bug Team:Elastic-Agent Label for the Agent team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants