Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix/dlq timestamp #7

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

robbavey
Copy link
Contributor

@robbavey robbavey commented Jun 14, 2017

Stop timestamp overriding committed offsets

Change behavior to only respect the 'startTimestamp' flag when no offsets have been committed.
Adds more tests.

Fixes #6

@robbavey robbavey force-pushed the fix/dlq_timestamp branch from 864add0 to 5d27419 Compare June 14, 2017 19:14
@@ -98,7 +101,6 @@ private void writeOffsets(Path segment, long offset) throws IOException {
}

public void close() throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor - can you add the @Override annotation ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will do

@@ -34,7 +34,7 @@
import java.util.function.Consumer;


public class DeadLetterQueueInputPlugin {
public class DeadLetterQueueInputPlugin implements AutoCloseable{
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor/style - since close is throws IOException this can implement Closeable instead of AutoClosable, which (imo) is a bit more idiomatic

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't have strong feelings either way - I added the AutoCloseable interface to allow the use of try.. with resources to help avoid resource leaks/clean up tests. My understanding was the AutoCloseable was more used for code targeting JDK7+, but I guess you are right in that the existing close() throws IOException implies Closeable. (That also means that I need to fix up the method to make it idempotent, which it should have been anyway... ;))

@ph
Copy link
Contributor

ph commented Sep 20, 2017

Its this PR related to fix the CI test on master and the 6.x branch?

https://travis-ci.org/logstash-plugins/logstash-input-dead_letter_queue

dead_letter_queue/src/test/java/org/logstash/input/DeadLetterQueueInputPluginTests.java:109: error: cannot find symbol
                targetDateString = entry.getEntryTime().toIso8601();
                                                       ^
  symbol:   method toIso8601()
  location: class Timestamp
1 error
:compileTestJava FAILED
:compileTestJava (Thread[main,5,main]) completed. Took 1.579 secs.

Change behavior to only respect the 'startTimestamp' flag when
no offsets have been committed.
@robbavey
Copy link
Contributor Author

@ph - the reason the tests are failing on 6.x and master is elastic/logstash#8128 (elastic/logstash@d121c58), which removes the toIso8601 method from timestamp.

This PR does remove that method, but a less invasive PR should probably go in between. I'll put it up shortly

@ph
Copy link
Contributor

ph commented Sep 20, 2017

Thanks @robbavey !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Dead letter queue input should only use timestamp as start date if no offset already stored.
3 participants