-
Notifications
You must be signed in to change notification settings - Fork 467
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feat (Core): PostgresJobQueue Implementation and Job Processing Enhancements #30175
Feat (Core): PostgresJobQueue Implementation and Job Processing Enhancements #30175
Commits on Sep 12, 2024
-
Configuration menu - View commit details
-
Copy full SHA for 40c342d - Browse repository at this point
Copy the full SHA 40c342dView commit details
Commits on Sep 13, 2024
-
#29479 Add job lifecycle events to JobQueueManagerAPI
Introduced multiple job lifecycle event classes including `JobCreatedEvent`, `JobStartedEvent`, `JobProgressUpdatedEvent`, `JobCompletedEvent`, `JobFailedEvent`, and `JobCancelledEvent`. Updated `JobQueueManagerAPIImpl` to fire these events at appropriate stages in the job lifecycle to enhance observability and event-driven processing. Updated tests to accommodate the changes in event handling.
Configuration menu - View commit details
-
Copy full SHA for 731b71c - Browse repository at this point
Copy the full SHA 731b71cView commit details -
#29479 Refactor job events to use Job objects directly
Replaced jobId strings with Job objects in event classes to simplify data access and reduce redundancy. Introduced RealTimeJobMonitor for handling real-time job updates, improving event notification efficiency and code maintainability. Updated tests and related classes to accommodate these changes.
Configuration menu - View commit details
-
Copy full SHA for acbceef - Browse repository at this point
Copy the full SHA acbceefView commit details -
#29479 Remove JobResult enum and refactor job handling logic
Remove the `JobResult` enum and update job processing methods to use an `AbstractJobResult` interface. This refactor centralizes job result handling, differentiates success, failure, and cancellation cases, and incorporates metadata and error details within the job result.
Configuration menu - View commit details
-
Copy full SHA for a15d7c8 - Browse repository at this point
Copy the full SHA a15d7c8View commit details
Commits on Sep 17, 2024
-
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for c8a91cd - Browse repository at this point
Copy the full SHA c8a91cdView commit details
Commits on Sep 18, 2024
-
#29479 Implement PostgresJobQueue and related error handling classes
This commit adds a PostgreSQL-specific implementation of the JobQueue interface, providing detailed methods for job management using a PostgreSQL database. It also introduces new error handling classes like JobQueueException, JobQueueDataException, and renames ProcessorNotFoundException to JobProcessorNotFoundException for clarity.
Configuration menu - View commit details
-
Copy full SHA for 7cd30ce - Browse repository at this point
Copy the full SHA 7cd30ceView commit details -
#29479 Refactor job queries to return paginated results.
Updated job queries to use common table expressions (CTEs) for pagination, ensuring all job retrieval methods now return a JobPaginatedResult object containing job data and pagination details. This improves the consistency and handling of job data across the application. Added new tests to verify the correctness of these changes.
Configuration menu - View commit details
-
Copy full SHA for 746b387 - Browse repository at this point
Copy the full SHA 746b387View commit details -
#29479 Refactor job mapping logic to separate utility class
Moved job result set mapping logic from `PostgresJobQueue` to new `DBJobTransformer` utility class. This change improves code modularity, readability, and maintainability by isolating the transformation logic in a dedicated class.
Configuration menu - View commit details
-
Copy full SHA for c7d6bcc - Browse repository at this point
Copy the full SHA c7d6bccView commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for 7def000 - Browse repository at this point
Copy the full SHA 7def000View commit details
Commits on Sep 19, 2024
-
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for 2807e1c - Browse repository at this point
Copy the full SHA 2807e1cView commit details -
Configuration menu - View commit details
-
Copy full SHA for 2955ab1 - Browse repository at this point
Copy the full SHA 2955ab1View commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for e7b3e1d - Browse repository at this point
Copy the full SHA e7b3e1dView commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for 4d0a0ea - Browse repository at this point
Copy the full SHA 4d0a0eaView commit details
Commits on Sep 20, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c66f1d0 - Browse repository at this point
Copy the full SHA c66f1d0View commit details -
Configuration menu - View commit details
-
Copy full SHA for 2a64551 - Browse repository at this point
Copy the full SHA 2a64551View commit details -
#29479 Add JobQueue getter to JobQueueManagerAPI
Implemented a public getter method for JobQueue in JobQueueManagerAPI. Updated corresponding tests to include assertions for the new method. This enhances testability and ensures the JobQueue dependency is correctly injected.
Configuration menu - View commit details
-
Copy full SHA for fd872ba - Browse repository at this point
Copy the full SHA fd872baView commit details
Commits on Sep 25, 2024
-
#29479 Refactor job cancellation handling
Decoupled job cancellation logic by introducing a `Cancellable` interface. This improves clarity and separation of concerns, ensuring only processors capable of cancellation implement the relevant method. Updated existing tests to support the new interface and ensured compatibility with the JobQueueManagerAPI.
Configuration menu - View commit details
-
Copy full SHA for a05c6ed - Browse repository at this point
Copy the full SHA a05c6edView commit details
Commits on Sep 26, 2024
-
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for 9b5c929 - Browse repository at this point
Copy the full SHA 9b5c929View commit details -
#29479 Refactor job cancellation handling
Decoupled job cancellation logic by introducing a `Cancellable` interface. This improves clarity and separation of concerns, ensuring only processors capable of cancellation implement the relevant method. Updated existing tests to support the new interface and ensured compatibility with the JobQueueManagerAPI.
Configuration menu - View commit details
-
Copy full SHA for 9053cae - Browse repository at this point
Copy the full SHA 9053caeView commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for a0c4da0 - Browse repository at this point
Copy the full SHA a0c4da0View commit details -
Add job cancellation handling and progress updates
This commit introduces a new `JobCancellingEvent` and related logic to handle job cancellations. It also refines the job state management and progress tracking to ensure accurate updates and status changes.
Configuration menu - View commit details
-
Copy full SHA for f6e181c - Browse repository at this point
Copy the full SHA f6e181cView commit details -
#29479 Refactor job state management and retry logic
Introduce markAsRunning() method to standardize setting job states to RUNNING. Replace usage of lastRetryTimestamp with completedAt for retry timing. Update tests to reflect changes and improve code readability. Ensure jobs are correctly re-queued with updated states.
Configuration menu - View commit details
-
Copy full SHA for 2b96c50 - Browse repository at this point
Copy the full SHA 2b96c50View commit details -
Configuration menu - View commit details
-
Copy full SHA for cb883de - Browse repository at this point
Copy the full SHA cb883deView commit details
Commits on Sep 27, 2024
-
Configuration menu - View commit details
-
Copy full SHA for c7ad2ec - Browse repository at this point
Copy the full SHA c7ad2ecView commit details -
Configuration menu - View commit details
-
Copy full SHA for af33445 - Browse repository at this point
Copy the full SHA af33445View commit details -
Configuration menu - View commit details
-
Copy full SHA for 079adbf - Browse repository at this point
Copy the full SHA 079adbfView commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for cb368bb - Browse repository at this point
Copy the full SHA cb368bbView commit details -
Configuration menu - View commit details
-
Copy full SHA for ada07ff - Browse repository at this point
Copy the full SHA ada07ffView commit details -
Configuration menu - View commit details
-
Copy full SHA for ec99927 - Browse repository at this point
Copy the full SHA ec99927View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0952bd7 - Browse repository at this point
Copy the full SHA 0952bd7View commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for 55ed75d - Browse repository at this point
Copy the full SHA 55ed75dView commit details -
Configuration menu - View commit details
-
Copy full SHA for 1afa6a1 - Browse repository at this point
Copy the full SHA 1afa6a1View commit details -
Configuration menu - View commit details
-
Copy full SHA for 0f1fb34 - Browse repository at this point
Copy the full SHA 0f1fb34View commit details
Commits on Sep 28, 2024
-
Configuration menu - View commit details
-
Copy full SHA for cccf9b3 - Browse repository at this point
Copy the full SHA cccf9b3View commit details -
Merge remote-tracking branch 'origin/master' into issue-29479-Create-…
…a-default-JobQueue-Implementation-using-Postgres
Configuration menu - View commit details
-
Copy full SHA for e5bb54a - Browse repository at this point
Copy the full SHA e5bb54aView commit details