Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add force_flush to LogExporter #2276

Open
wants to merge 14 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions opentelemetry-sdk/src/export/logs/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,25 @@ pub trait LogExporter: Send + Sync + Debug {
// By default, all logs are enabled
true
}

/// This is a hint to ensure that the export of any Logs the exporter
/// has received prior to the call to this function SHOULD be completed
/// as soon as possible, preferably before returning from this method.
///
/// This function SHOULD provide a way to let the caller know
/// whether it succeeded, failed or timed out.
///
/// This function SHOULD only be called in cases where it is absolutely necessary,
/// such as when using some FaaS providers that may suspend the process after
/// an invocation, but before the exporter exports the completed logs.
///
/// This function SHOULD complete or abort within some timeout. This function can be
Copy link
Contributor

@utpilla utpilla Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's update the comments here to reflect the actual trait method definition instead of putting the spec guidelines for this method.

We could put something like:

/// This method will block the current thread until all pending log records are exported. If the export was not
/// successful, an error is returned.

/// implemented as a blocking API or an asynchronous API which notifies the caller via
/// a callback or an event. OpenTelemetry client authors can decide if they want to
/// make the flush timeout configurable.
fn force_flush(&mut self) -> ExportResult {
Ok(())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should not have a default implementation for this method. Otherwise, we could run into a situation where an exporter user goes through the documentation and calls force_flush with certain expectations that haven't been agreed upon by the exporter author.

It looks like we have a default implementation for a few other methods as well such as shutdown and set_resource. We should revisit them based on what we decide.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Otherwise, we could run into a situation where an exporter user goes through the documentation and calls force_flush with certain expectations that haven't been agreed upon by the exporter author.

If the exporter author does not provide an implementation for flush, then they can document that right? Or the doc for Provider can be updated to merely state that it'll invoke flush on the processor, and the same for processor can state they will invoke the flush for exporter.

Default implementation seems reasonable to me, as not every exporter need to do something for flush.

}
/// Set the resource for the exporter.
fn set_resource(&mut self, _resource: &Resource) {}
}
Expand Down
29 changes: 26 additions & 3 deletions opentelemetry-sdk/src/logs/log_processor.rs
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,11 @@
}

fn force_flush(&self) -> LogResult<()> {
Ok(())
if let Ok(mut exporter) = self.exporter.lock() {
exporter.force_flush()
} else {
Err(LogError::MutexPoisoned("SimpleLogProcessor".into()))

Check warning on line 135 in opentelemetry-sdk/src/logs/log_processor.rs

View check run for this annotation

Codecov / codecov/patch

opentelemetry-sdk/src/logs/log_processor.rs#L135

Added line #L135 was not covered by tests
}
}

fn shutdown(&self) -> LogResult<()> {
Expand Down Expand Up @@ -278,7 +282,8 @@
&timeout_runtime,
logs.split_off(0),
)
.await;
.await
.and(exporter.as_mut().force_flush());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this? export_with_timeout already calls exporter.export which is what is expected from force_flush.


if let Some(channel) = res_channel {
if let Err(send_error) = channel.send(result) {
Expand Down Expand Up @@ -803,6 +808,25 @@
let _ = provider.shutdown();
}

#[tokio::test(flavor = "multi_thread")]
async fn test_batch_forceflush() {
let exporter = InMemoryLogExporterBuilder::default().build();
// TODO: Verify exporter.force_flush() is called

let processor = BatchLogProcessor::new(
Box::new(exporter.clone()),
BatchConfig::default(),
runtime::Tokio,
);

let mut record = LogRecord::default();
let instrumentation = InstrumentationScope::default();

processor.emit(&mut record, &instrumentation);
processor.force_flush().unwrap();
assert_eq!(1, exporter.get_emitted_logs().unwrap().len());
}

#[tokio::test(flavor = "multi_thread")]
async fn test_batch_shutdown() {
// assert we will receive an error
Expand All @@ -820,7 +844,6 @@
let instrumentation = InstrumentationScope::default();

processor.emit(&mut record, &instrumentation);
processor.force_flush().unwrap();
processor.shutdown().unwrap();
// todo: expect to see errors here. How should we assert this?
processor.emit(&mut record, &instrumentation);
Expand Down
Loading