Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix: docker compose pip dependencies #36123

Closed
wants to merge 17 commits into from
Closed

Fix: docker compose pip dependencies #36123

wants to merge 17 commits into from

Conversation

kuisathaverat
Copy link
Contributor

@kuisathaverat kuisathaverat commented Jul 20, 2023 •
edited
Loading

What does this PR do?

Ti changes the version of PyYAML to 5.4.1 due to dependency conflicts with docker-compose 1.29.2. The version of PyYAML must be 6< to fix the conflict.

Why is it important?

Several targets are broken because of this conflict.

Checklist

  • [ ] My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • [ ] I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Prerequisites

GO_VERSION=$(cat .go-version)
gvm install ${GO_VERSION}
eval $(gvm use ${GO_VERSION})

PYENV_VERSION=$(cat .python-version)
pyenv install ${PYENV_VERSION}
make -C beats/x-pack/heartbeat/ironbank package
mage -d metricbeat goIntegTest
mage -d metricbeat pythonIntegTest
mage -d x-pack/metricbeat goIntegTest
mage -C x-pack/metricbeat pythonIntegTest
make -C metricbeat check MAGEFILE_VERBOSE=1
make -C x-pack/metricbeat check MAGEFILE_VERBOSE=1
make -C metricbeat check MAGEFILE_VERBOSE=1
make -C x-pack/metricbeat check MAGEFILE_VERBOSE=1
make -C metricbeat  build unitTest MAGEFILE_VERBOSE=1 
make -C x-pack/metricbeat  build unitTest MAGEFILE_VERBOSE=1

reakaleek and others added 3 commits July 19, 2023 16:59
@reakaleek
Downgrade PyYaml to 5.3.1
80ce812 
Upgrading to 6.0.1 did not work because there is a dependency conflict with docker-compose.

When using the version 5.4.1 the following error message appears when trying to run `pip install -r requirements.txt`

```
AttributeError: cython_source
```

Related issue: yaml/pyyaml#601
@kuisathaverat
fix: Ironbank package
d1237fb
@kuisathaverat
fix: pip dependency broken
a578e3d
@kuisathaverat kuisathaverat requested a review from a team as a code owner July 20, 2023 07:56
@kuisathaverat kuisathaverat requested review from belimawr and faec and removed request for a team July 20, 2023 07:56
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Jul 20, 2023
@botelastic
Copy link

botelastic bot commented Jul 20, 2023

This pull request doesn't have a Team:<team> label.

@mergify mergify bot mentioned this pull request Jul 20, 2023
Closed
@elasticmachine
Copy link
Collaborator

elasticmachine commented Jul 20, 2023 •
edited by jenkins-beats-ci bot
Loading

💔 Build Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2023-07-21T11:19:08.185+0000

  • Duration: 45 min 4 sec

Test stats 🧪

Test Results
Failed 0
Passed 818
Skipped 254
Total 1072

Steps errors 2

Expand to view the steps failures

metricbeat-packaging-arm-ubuntu-2204-aarch64 - mage package
  • Took 2 min 21 sec . View more details here
  • Description: mage package
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

Expand to view the GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@kuisathaverat kuisathaverat force-pushed the fix/docker-compose-pip-dependencies branch from 219f573 to 9d4d089 Compare July 20, 2023 08:47
@kuisathaverat kuisathaverat requested review from platform-ingest and a team July 20, 2023 09:10
@kuisathaverat
Copy link
Contributor Author

The test that fails needs large runners; the default ones with 8GB are not enough (about 130 tests of 800).
The Windows test needs Linux containers, so it requires a self-hosted runner with support for Linux containers. Github runners support only Windows containers.
Everything else works fine.

@mergify
Copy link
Contributor

mergify bot commented Aug 1, 2023

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b fix/docker-compose-pip-dependencies upstream/fix/docker-compose-pip-dependencies
git merge upstream/8.8
git push upstream fix/docker-compose-pip-dependencies

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@belimawr belimawr Awaiting requested review from belimawr belimawr is a code owner automatically assigned from elastic/elastic-agent-data-plane

@faec faec Awaiting requested review from faec faec is a code owner automatically assigned from elastic/elastic-agent-data-plane

@platform-ingest platform-ingest Awaiting requested review from platform-ingest

Assignees

@kuisathaverat kuisathaverat

Labels
needs_team Indicates that the issue/PR needs a Team:* label
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@kuisathaverat @elasticmachine @reakaleek