-
Notifications
You must be signed in to change notification settings - Fork 751
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GOBBLIN-1870] Fix missing flow execution id bug causing SQL Error #3733
Conversation
Codecov Report
@@ Coverage Diff @@
## master #3733 +/- ##
============================================
+ Coverage 45.34% 47.10% +1.76%
- Complexity 2138 10862 +8724
============================================
Files 413 2146 +1733
Lines 17956 84783 +66827
Branches 2184 9409 +7225
============================================
+ Hits 8142 39939 +31797
- Misses 8947 41221 +32274
- Partials 867 3623 +2756
... and 1734 files with indirect coverage changes 📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
log.info("Multi-active scheduler about to handle trigger event: [{}, triggerEventTimestamp: {}]", flowAction, | ||
eventTimeMillis); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm all for whatever logging is needed to understand and debug. just wondering here: couldn't essentially similar logging instead live within tryAcquireLease
, which is basically what this message announces is about to be invoked?
e.g. are there too many code paths leading to tryAcquireLease
, yet no good way within the method to pinpoint that it was specifically this path?
@@ -244,6 +244,7 @@ public void orchestrate(Spec spec, Properties jobProps, long triggerTimestampMil | |||
return; | |||
} | |||
Map<String, String> flowMetadata = TimingEventUtils.getFlowMetadata((FlowSpec) spec); | |||
FlowCompilationValidationHelper.addFlowExecutionIdIfAbsent(flowMetadata, jobExecutionPlanDagOptional.get()); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
seems an important property to add.
on that note, do we have any existing Orchestrator.orchestrate
test case we might be able to augment with an assert that this is in fact set? (e.g. using test inputs that would fail that assert w/o the above code change.)
…3733) Co-authored-by: Urmi Mustafi <[email protected]>
Dear Gobblin maintainers,
Please accept this PR. I understand that it will not be reviewed until I have checked off all the steps below!
JIRA
Description
Seeing the following issue when trying to acquire lease for a flow action:
java.sql.SQLIntegrityConstraintViolationException: Column 'flow_execution_id' cannot be null"
Flow action is created in orchestrate function, during refactoring missed executing the line to add flow execution id to metadata to be extracted for multi-active case. Adding a log line as well to check the flow action object created and verify it has all the flow information.
Tests
Commits