Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ingest order sorting algorithm does not traverse deep enough when a single CSV contains 4 levels and just UUIDs #201

Open
DiegoPino opened this issue May 10, 2024 · 2 comments
Open

Ingest order sorting algorithm does not traverse deep enough when a single CSV contains 4 levels and just UUIDs #201

DiegoPino opened this issue May 10, 2024 · 2 comments
Assignees
@DiegoPino
Labels
AMI Import Plugins Gets data from somewhere, puts data into a strawberry basked bug Something isn't working CSV Processing Things we do here and there to keep the tabulated goddesses happy queue workers Ones taking the FI and doing the FO queue FIFO
Milestone
0.9.0

Comments

@DiegoPino
Copy link
Member

What?

ON AMI ingest we presort the ADOs to ensure Top level/parents get ingested first. But we don't traverse deep enough when sorting (specially when using UUIDs for the semantic relationships) leading to the queue having children first. We also have a safety feature for that (a retry) but on very heavily nested structures, the number of retries is not enough

The task is to make the Sorting by parent algorithm faster/better and maybe just as failsafe add an extra retry

@alliomeria as observed with scrapbooks
@DiegoPino DiegoPino added bug Something isn't working queue FIFO queue workers Ones taking the FI and doing the FO AMI Import Plugins Gets data from somewhere, puts data into a strawberry basked CSV Processing Things we do here and there to keep the tabulated goddesses happy labels May 10, 2024
@DiegoPino DiegoPino added this to the 0.8.0 milestone May 10, 2024
@DiegoPino DiegoPino self-assigned this May 10, 2024
@DiegoPino DiegoPino modified the milestones: 0.8.0, 0.9.0 Aug 15, 2024
@DiegoPino
Copy link
Member Author

Still and issue .. .

@DiegoPino
Copy link
Member Author

still an issue (@alliomeria 🍳 )

@DiegoPino DiegoPino mentioned this issue Nov 8, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DiegoPino DiegoPino

Labels
AMI Import Plugins Gets data from somewhere, puts data into a strawberry basked bug Something isn't working CSV Processing Things we do here and there to keep the tabulated goddesses happy queue workers Ones taking the FI and doing the FO queue FIFO
Projects
None yet
Milestone
0.9.0
Development

No branches or pull requests
1 participant
@DiegoPino