Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a git distributor in dmci #191

Open
johtoblan opened this issue Aug 14, 2023 · 2 comments
Open

Create a git distributor in dmci #191

johtoblan opened this issue Aug 14, 2023 · 2 comments
Assignees

Comments

@johtoblan
Copy link
Collaborator

johtoblan commented Aug 14, 2023

Use Gitlab-api to push incoming mmd-files. In this way, we may not need the file distributor which requires a local copy of the mmd-xml archive, and pushing of an updated copy at regular intervals (currently 1 hour).

@mortenwh
Copy link
Collaborator

mortenwh commented Jan 19, 2024

Is it correct that this was done because the local folder got full? Now we have another problem that commits take very long time - up to an hour to finish (40 minutes between 213d9f73 and 7705b10c for 4 new files). That apparently hampers the ability to merge in MRs. Two MRs (https://gitlab.met.no/mmd/mmd-xml-dev/-/merge_requests/21, and https://gitlab.met.no/mmd/mmd-xml-dev/-/merge_requests/23) are not possible to merge (on 19th January 2024). After pushing the merge button they just return to "ready to merge". Maybe better to revert to the old setup and make a large volume then..?

@mortenwh mortenwh reopened this Jan 19, 2024
@johtoblan
Copy link
Collaborator Author

johtoblan commented Jan 22, 2024

The problem was not only local storage, but how the git-pusher handled interruptions in the increasingly slow indexing of the whole archive everytime we wanted to push. This was to be solved by removing the slow indexing.

@mortenwh mortenwh changed the title Use Gitlab-api to push incoming mmd-files, instead of having a local copy of the mmd-xml archive and pushing an updated copy Create a git distributor in dmci Jan 25, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants