Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

UDP based job manager #604

Closed
jdries opened this issue Aug 19, 2024 · 4 comments
Closed

UDP based job manager #604

jdries opened this issue Aug 19, 2024 · 4 comments
Assignees
@soxofaan
Labels
job manager

Comments

@jdries
Copy link
Collaborator

jdries commented Aug 19, 2024 •
edited
Loading

Support a job manager that automatically creates jobs based on the schema of a UDP.

Prototype is already available here, with some valid comments that should be taken into account:
ESA-APEx/esa-apex-toolbox-python#1
@JeroenVerstraelen JeroenVerstraelen self-assigned this Aug 19, 2024
@JeroenVerstraelen
Copy link
Contributor

There is a PR, just needs small adjustments.

@jdries jdries linked a pull request Oct 1, 2024 that will close this issue
add job manager that works on a single (user-defihed) process #621
Closed
soxofaan added a commit that referenced this issue Oct 10, 2024
@soxofaan
Issue #604 initial PoC implementation of UDPJobFactory
e4eedd6
@soxofaan soxofaan mentioned this issue Oct 10, 2024
Merged
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 refactor out parse_remote_process_definition as stand…
607f78e 
…alone utility

Finetune openeo.internal.processes.parse to properly support this
(e.g. leverage named tuples for immutability, less boilerplate and out of the box equality checks)
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 explicitly use default value from schema as fallback
a9e94d2 
improves long term traceability and observability
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 add UDPJobFactory+MultiBackendJobManager tests
31d0c76
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604 initial PoC implementation of UDPJobFactory
ec982c2
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 refactor out parse_remote_process_definition as stand…
4e04351 
…alone utility

Finetune openeo.internal.processes.parse to properly support this
(e.g. leverage named tuples for immutability, less boilerplate and out of the box equality checks)
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 explicitly use default value from schema as fallback
04bc791 
improves long term traceability and observability
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 add UDPJobFactory+MultiBackendJobManager tests
ff8b553
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 more UDPJobFactory+MultiBackendJobManager tests
04296c1
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 UDPJobFactory: improve geometry support
f8db877
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 test coverage for personal UDP mode
ff9c3f2 
and some related refactoring
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 test coverage for geometry handling after resume
632e239
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 test coverage for resuming: also parquet
fd9fdb8
soxofaan added a commit that referenced this issue Oct 11, 2024
@soxofaan
Issue #604/#644 fix title/description
ade5258
@soxofaan
Copy link
Member

worked on PR at #644

soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 changelog entry and usage example
d29d3ee
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 add process_id to docs
a92e47f
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 UDPJobFactory: make process_id optional (if namespace…
84ee8ea 
… is given)
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 replace lru_cache trick with cleaner cache
ddc9ee5
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
fixup! Issue #604/#644 add process_id to docs
0f7a9bc
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 further documentation finetuning
e22a791
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 add tests for parameter_column_map
130db87
soxofaan added a commit that referenced this issue Oct 14, 2024
@soxofaan
Issue #604/#644 add some todo note for furtherdevelopment ideas
2667733
@soxofaan
Copy link
Member

#644 is ready for review/mering I think

soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
Issue #604/#644 finetune docs based on review
1917a73
soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
Issue #604/#644 Rename UDPJobFactory to ProcessBasedJobCreator
8ffa2a6
soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
Issue #604/#644 move ProcessBasedJobCreator example to more extensive…
f09efe0 
… doc page
soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
fixup! Issue #604/#644 move ProcessBasedJobCreator example to more ex…
5c14963 
…tensive doc page
soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
fixup! fixup! Issue #604/#644 move ProcessBasedJobCreator example to …
2030fac 
…more extensive doc page
soxofaan added a commit that referenced this issue Oct 16, 2024
@soxofaan
Issue #604/#644 move ProcessBasedJobCreator example to more extensive…
7bf73de 
… doc page
@soxofaan
Copy link
Member

merged #644

@soxofaan soxofaan mentioned this issue Oct 16, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@soxofaan soxofaan

Labels
job manager
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

add job manager that works on a single (user-defihed) process Open-EO/openeo-python-client
3 participants
@soxofaan @JeroenVerstraelen @jdries