[issue-265] Reader side: Implement watermark with Pravega (for master branch aka. Flink 1.9) #279
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
crazyzhou
changed the title
* Implement reader with watermark * fix Table API * Use SerializedValue to fix the serialization error, add a basic test * Add check for watermark, Fix test * Bump pravega version to dceb098 * use bigger heap * Produce better messages if test fails * Added comment to LowerBoundAssigner Signed-off-by: Brian Zhou <[email protected]>
crazyzhou
force-pushed
the
feature-watermark
branch
from
4e03451 to
279e958
Compare
Codecov Report
@@ Coverage Diff @@
## master #279 +/- ##
============================================
+ Coverage 84.44% 84.52% +0.07%
- Complexity 322 337 +15
============================================
Files 34 35 +1
Lines 1466 1544 +78
Branches 156 168 +12
============================================
+ Hits 1238 1305 +67
- Misses 122 129 +7
- Partials 106 110 +4
Continue to review full report at Codecov.
|
crazyzhou
commented
Oct 17, 2019
| * @param assignerWithTimeWindows the timestamp and watermark assigner. | ||
| * @return TableSourceReaderBuilder instance. | ||
| */ | ||
| // TODO: Due to the serialization validation for `connectorProperties`, only `public` `static-inner/outer` class implements |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change log description
Purpose of the change
Same as the pull request #274 for
r0.6-flink1.7, themasterbranch require only merging PRs rather than cherry-picking, I create this PR for the master branch.What the code does
Same as PR #274 , modified only
ClosureCleaner.clean(obj, bool)which is removed since Flink 1.8 intoClosureCleaner.clean(obj, ExecutionConfig.ClosureCleanerLevel.RECURSIVE, bool)TableValidationUtilwhich is removed since Flink 1.9 intoTableSourceValidation