Install llnl-scraper from the LLNL/scraper GitHub repository
#147
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This includes updating all of the system packages installed to the versions available for Alpine Linux 3.20.
This updates the `build-stage` to match the versions of Python and Alpine Linux used in the `compile-stage`. This also includes updating the versions of all system packages installed.
Bump the Python packages installed in the `compile-stage` as follows: - pip from 23.1.2 to 24.2 - pipenv from 2023.10.20 to 2024.0.1 - setuptools from 67.7.2 to 73.0.1 - wheel from 0.40.0 to 0.44.0
Update the version of Python declared in the Pipfile and update the dependencies installed in the Python virtual environment by running `pipenv lock` in the `src/` directory.
Alpine Linux 3.20 configures Python 3 as externally managed, so we need to pass this flag to `pip` to install a Python package directly. Since we are using this to build a Python virtual environment in the `compile-stage` that is moved to the `build-stage` for use in the final image this should pose no issues.
We need some changes that were added in the LLNL/scraper repo but do not yet have a PyPI release. Rather than continue to use a fork in our GitHub organization it makes sense to pull in LLNL/scraper at a specific commit until a new version is release to PyPI.
mcdonnnj
added
bug
This was referenced Aug 29, 2024
dav3r
approved these changes
Aug 29, 2024
dv4harr10
approved these changes
Aug 29, 2024
jsf9k
approved these changes
Aug 29, 2024
8 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
🗣 Description
This pull request drops the usage of the cisagov/scraper project and reverts back to using the official LLNL/scraper project.
Note
This pull request is built on top of #146.
💭 Motivation and context
Originally the cisagov/scraper project was made and used to get some bugfix code into this project. Now that repository is behind the upstream LLNL/scraper project which includes needed bugfix code. Rather than keep maintaining the cisagov/scraper project it makes sense to revert to the LLNL/scraper project. However, since the bugfix code we need is not available in a PyPI release for the
llnl-scraperpackage, we have to pull directly from the LLNL/scraper repository. So we are pulling the same code consistently we pin to the latest commit on the default branch, but we should switch to pulling from PyPI when a new version is available.🧪 Testing
Testing for this pull request is in the same state as that for #146 where the final image build is timing out.
✅ Pre-approval checklist